Image encoding/decoding method and device

ABSTRACT

The present invention relates to an image encoding/decoding method and device. In an image decoding method and device according to an embodiment of the present invention, a reconstructed pixel region within an image to which a current block to be decoded belongs is selected; a motion vector of the reconstructed pixel region is derived on the basis of the reconstructed pixel region and a reference image of the current block; and the derived motion vector is selected as a motion vector of the current block.

TECHNICAL FIELD

The present invention relates to an image signal encoding/decodingmethod and device. More particularly, the present invention relates toan image encoding/decoding method using inter prediction and an imageencoding/decoding device using inter prediction.

BACKGROUND ART

Recently, demand for multimedia data such as video has rapidly increasedon the Internet. However, the rate at which a bandwidth of a channel hasdeveloped is insufficient to keep up with the amount of multimedia datathat has rapidly increased. Considering this situation, the Video CodingExpert Group (VCEG) of ITU-T and the Moving Picture Expert Group (MPEG)of ISO/IEC, which are the International Organizations forStandardization, established the High Efficiency Video Coding (HEVC)version 1, a video compression standard, in February 2014.

HEVC uses a variety of technologies such as intra prediction, interprediction, transform, quantization, entropy encoding, and in-loopfiltering. In inter prediction of HEVC, new technologies such as blockmerging, advanced motion vector prediction (AMVP) have been applied suchthat efficient inter prediction is possible. However, when multiplemotions are present in a block, the block is partitioned into smallparts, so that rapid increase in overhead may occur and encodingefficiency may be lowered.

DISCLOSURE Technical Problem

Accordingly, the present invention has been made keeping in mind theabove problems, and the present invention is intended to enhanceefficiency of inter prediction by providing improved inter prediction.

Also, the present invention is intended to provide a motion vectorderivation method by an image decoding device, where an image encodingdevice does not need to transmit motion vector information to the imagedecoding device.

Also, the present invention is intended to provide a motion vectorderivation method of a control point by an image decoding device,wherein in affine inter prediction, an image encoding device does notneed to transmit a motion vector of the control point to the imagedecoding device.

Also, the present invention is intended to provide inter predictioncapable of efficient encoding or decoding when multiple motions arepresent in one block.

Also, the present invention is intended to reduce blocking artifactsthat may occur when one block is partitioned into multiple regions andencoding or decoding is performed using different types of interprediction.

Also, the present invention is intended to enhance efficiency of interprediction by partitioning a current block to be encoded or decoded, onthe basis of a partitioning structure of reconstructed neighboringblocks.

Also, the present invention is intended to enhance efficiency of interprediction by partitioning, on the basis of a partitioning structure ofreconstructed neighboring blocks, a pre-reconstructed neighboring imageregion which is used to encode or decode a current block.

Also, the present invention is intended to enhance efficiency of imageencoding or decoding by performing encoding or decoding using a currentblock or a neighboring image partitioned as described above.

Technical Solution

In an image decoding method and device according to an embodiment of thepresent invention, a reconstructed pixel region within an image to whicha current block to be decoded belongs is selected; on the basis of thereconstructed pixel region and a reference image of the current block, amotion vector of the reconstructed pixel region is derived; and thederived motion vector is selected as a motion vector of the currentblock.

The reconstructed pixel region may include at least one among a regionadjacent to an upper side of the current block and a region adjacent toa left side of the current block.

The motion vector of the reconstructed pixel region may be derived onthe basis of a position of a region corresponding to the reconstructedpixel region, wherein the region corresponding to the reconstructedpixel region is determined within the reference image.

In an image encoding method and device according to an embodiment of thepresent invention, a reconstructed pixel region within an image to whicha current block to be encoded belongs is selected; on the basis of thereconstructed pixel region and a reference image of the current block, amotion vector of the reconstructed pixel region is derived; and thederived motion vector is selected as a motion vector of the currentblock.

Also, in the image encoding method and device according to theembodiment of the present invention, decoder-side motion vectorderivation indication information may be generated and encoded.

The decoder-side motion vector derivation indication information mayindicate whether or not the derived motion vector of the reconstructedpixel region is selected as the motion vector of the current block.

In an image decoding method and device according to another embodimentof the present invention, at least one reconstructed pixel region isselected within an image to which a current block to be decoded usingaffine inter prediction belongs; on the basis of the at least onereconstructed pixel region and a reference image of the current block, amotion vector of the at least one reconstructed pixel region is derived;and the derived motion vector of the at least one reconstructed pixelregion is selected as a motion vector of at least one control point ofthe current block.

The at least one reconstructed pixel region may be a region adjacent tothe at least one control point of the current block.

Further, the at least one control point may be positioned at an upperleft side, an upper right side, or a lower left side of the currentblock.

Further, the motion vector of the control point positioned at a lowerright side of the current block may be decoded on the basis of motioninformation included in a bitstream.

Further, in the image decoding method and device according to theembodiment of the present invention, decoder-side control point motionvector derivation indication information may be decoded.

In the image decoding method and device according to the embodiment ofthe present invention, the motion vector of the at least onereconstructed pixel region may be derived on the basis of thedecoder-side control point motion vector derivation indicationinformation.

In an image encoding method and device according to still anotherembodiment of the present invention, at least one reconstructed pixelregion is selected within an image to which a current block to beencoded using affine inter prediction belongs; on the basis of the atleast one reconstructed pixel region and a reference image of thecurrent block, a motion vector of the at least one reconstructed pixelregion is derived; and the derived motion vector of the at least onereconstructed pixel region is selected as a motion vector of at leastone control point of the current block.

In an image decoding method and device according to yet still anotherembodiment of the present invention, a current block to be decoded ispartitioned into multiple regions including a first region and a secondregion; and a prediction block of the first region and a predictionblock of the second region are obtained, wherein the prediction block ofthe first region and the prediction block of the second region areobtained by different inter prediction methods.

The first region may be a region adjacent to a reconstructed imageregion within an image to which the current block belongs, and thesecond region may be a region that is not in contact with thereconstructed image region within the image to which the current blockbelongs.

In the image decoding method and device according to the embodiment ofthe present invention, on the basis of the reconstructed image regionwithin the image to which the current block belongs, and of a referenceimage of the current block, a motion vector of the first region may beestimated.

In the image decoding method and device according the embodiment of thepresent invention, a region positioned at a boundary as a region withinthe prediction block of the first region or a region positioned at aboundary as a region within the prediction block of the second regionmay be partitioned into multiple sub-blocks; motion information of aneighboring sub-block of a first sub-block, which is one of the multiplesub-blocks, may be used to generate a prediction block of the firstsub-block; and the first sub-block and the prediction block of the firstsub-block may be subjected to a weighted sum, so that a prediction blockof the first sub-block to which the weighted sum is applied may beobtained.

In an image encoding method and device according to yet still anotherembodiment of the present invention, a current block to be encoded ispartitioned into multiple regions including a first region and a secondregion; a prediction block of the first region and a prediction block ofthe second region are obtained, wherein the prediction block of thefirst region and the prediction block of the second region are obtainedby different inter prediction methods.

The first region may be a region adjacent to a pre-encoded reconstructedimage region within an image to which the current block belongs, and thesecond region may be a region that is not in contact with thepre-encoded reconstructed image region within the image to which thecurrent block belongs.

In the image encoding method and device according to the embodiment ofthe present invention, on the basis of the pre-encoded reconstructedimage region within the image to which the current block belongs, and ofa reference image of the current block, a motion vector of the firstregion may be estimated.

In the image encoding method and device according to the embodiment ofthe present invention, a region positioned at a boundary as a regionwithin the prediction block of the first region or a region positionedat a boundary as a region within the prediction block of the secondregion may be partitioned into multiple sub-blocks; motion informationof a neighboring sub-block of a first sub-block, which is one of themultiple sub-blocks, may be used to generate a prediction block of thefirst sub-block; and the first sub-block and the prediction block of thefirst sub-block may be subjected to a weighted sum, so that a predictionblock of the first sub-block to which the weighted sum is applied may beobtained.

In an image decoding method and device according to yet still anotherembodiment of the present invention, on the basis of blocks around acurrent block to be decoded, the current block is partitioned intomultiple sub-blocks, and the multiple sub-blocks of the current blockare decoded.

In the image decoding method and device according to the embodiment ofthe present invention, on the basis of a partitioning structure ofneighboring blocks of the current block, the current block may bepartitioned into the multiple sub-blocks.

In the image decoding method and device according to the embodiment ofthe present invention, on the basis of at least one among the number ofthe neighboring blocks, a size of the neighboring blocks, a shape of theneighboring blocks, and a boundary between the neighboring blocks, thecurrent block may be partitioned into the multiple sub-blocks.

In the image decoding method and device according to the embodiment ofthe present invention, as a region neighbors the current block, apre-reconstructed pixel region may be partitioned on a per-sub-blockbasis, and at least one of the multiple sub-blocks of the current blockmay be decoded using at least one sub-block included in thereconstructed pixel region.

In the image decoding method and device according to the embodiment ofthe present invention, on the basis of a partitioning structure ofneighboring blocks of the current block, the reconstructed pixel regionmay be partitioned on a per-sub-block basis.

In the image decoding method and device according to the embodiment ofthe present invention, on the basis of at least one among the number ofthe neighboring blocks, a size of the neighboring blocks, a shape of theneighboring blocks, and a boundary between the neighboring blocks, thereconstructed pixel region may be partitioned on a per-sub-block basis.

In an image encoding method and device according to yet still anotherembodiment of the present invention, on the basis of blocks around acurrent block to be encoded, the current block may be partitioned intomultiple sub-blocks, and the multiple sub-blocks of the current blockmay be encoded.

In the image encoding method and device according to the embodiment ofthe present invention, on the basis of a partitioning structure ofneighboring blocks of the current block, the current block may bepartitioned into the multiple sub-blocks.

In the image encoding method and device according to the embodiment ofthe present invention, on the basis of the at least one among the numberof the neighboring blocks, a size of the neighboring blocks, a shape ofthe neighboring blocks, and a boundary between the neighboring blocks,the current block may be partitioned into the multiple sub-blocks.

In the image encoding method and device according to the embodiment ofthe present invention, as a region neighbors the current block, apre-reconstructed pixel region may be partitioned on a per-sub-blockbasis, and at least one of the multiple sub-blocks of the current blockmay be encoded using at least one sub-block included in thereconstructed pixel region.

In the image encoding method and device according to the embodiment ofthe present invention, on the basis of a partitioning structure ofneighboring blocks of the current block, the reconstructed pixel regionmay be partitioned on a per-sub-block basis.

In the image encoding method and device according to the embodiment ofthe present invention, on the basis of at least one among the number ofthe neighboring blocks, a size of the neighboring blocks, a shape of theneighboring blocks, and a boundary between the neighboring blocks, thereconstructed pixel region may be partitioned on a per-sub-block basis.

Advantageous Effects

According to the present invention, the amount of encoding informationgenerated as a result of encoding a video may be reduced and thusencoding efficiency may be enhanced. Also, by adaptively decoding anencoded image, reconstruction efficiency of an image may be enhanced andthe quality of the reproduced image may be improved.

Also, in inter prediction according to the present invention, the imageencoding device does not need to transmit motion vector information tothe image decoding device, so that the amount of encoding informationmay be reduced and thus encoding efficiency may be enhanced.

Also, according to the present invention, blocking artifacts may bereduced that may occur when one block is partitioned into multipleregions and encoding or decoding is performed using different types ofinter prediction.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an image encoding deviceaccording to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a method of generating motioninformation by using motion estimation according to the conventionaltechnology.

FIG. 3 is a diagram illustrating an example of neighboring blocks thatmay be used to generate motion information of a current block.

FIG. 4 is a block diagram illustrating an image decoding deviceaccording to an embodiment of the present invention.

FIGS. 5a and 5b are diagrams illustrating inter prediction using areconstructed pixel region according to a first exemplary embodiment ofthe present invention.

FIG. 6 is a flowchart illustrating an inter prediction method accordingto the first exemplary embodiment of the present invention.

FIGS. 7a to 7c are diagrams illustrating examples of reconstructed pixelregions.

FIG. 8 is a flowchart illustrating a process of determining an interprediction method according to an embodiment of the present invention.

FIG. 9 is a diagram illustrating a process of encoding information thatindicates an inter prediction method.

FIG. 10 is a diagram illustrating a process of decoding DMVD indicationinformation encoded as shown in FIG. 9.

FIG. 11 is a diagram illustrating affine inter prediction.

FIGS. 12a and 12b are diagrams illustrating derivation of a motionvector of a control point by using a reconstructed pixel regionaccording to a second exemplary embodiment of the present invention.

FIG. 13 is a flowchart illustrating an inter prediction method accordingto the second exemplary embodiment of the present invention.

FIG. 14 is a flowchart illustrating a process of determining an interprediction method according to the second exemplary embodiment of thepresent invention.

FIG. 15 is a diagram illustrating a process of encoding information thatindicates an inter prediction method determined by the process shown inFIG. 14.

FIG. 16 is a diagram illustrating a process of decoding DCMVD indicationinformation encoded as shown in FIG. 15.

FIG. 17 is a flowchart illustrating an example of an image decodingmethod in which motion vectors of three control points are derived usinga reconstructed pixel region so as to generate a prediction block of acurrent block.

FIG. 18 is a diagram illustrating a current block partitioned intomultiple regions for inter prediction according to a third exemplaryembodiment of the present invention.

FIG. 19 is a flowchart illustrating an inter prediction method accordingto the third exemplary embodiment of the present invention.

FIG. 20 is a diagram illustrating an example of motion estimation andmotion compensation using a reconstructed pixel region.

FIG. 21 is a flowchart illustrating a process of determining an interprediction method according to an embodiment of the present invention.

FIG. 22 is a diagram illustrating a process of transmitting, to an imagedecoding device, information that indicates an inter prediction method.

FIG. 23 is a diagram illustrating a process of decoding information thatindicates which type of inter prediction has been used.

FIG. 24 is a flowchart illustrating a method of generating an interprediction block by using information that indicates which type of interprediction has been used.

FIG. 25 is a diagram illustrating a method of reducing blockingartifacts according to a fourth exemplary embodiment of the presentinvention.

FIG. 26 is a diagram illustrating a method of applying a weighted sum ofa sub-block within a prediction block and a sub-block adjacent to theupper side thereof.

FIG. 27 is a diagram illustrating a method of applying a weighted sum ofa sub-block within a prediction block and a sub-block adjacent to theleft side thereof.

FIG. 28 is a flowchart illustrating a process of determining whether ornot a weighted sum is applied between sub-blocks.

FIG. 29 is a flowchart illustrating a process of encoding informationthat indicates whether or not a weighted sum is applied betweensub-blocks.

FIG. 30 is a flowchart illustrating a process of decoding informationthat indicates whether or not a weighted sum is applied betweensub-blocks.

FIGS. 31a and 31b are diagrams illustrating inter prediction using areconstructed pixel region according to a fifth exemplary embodiment ofthe present invention.

FIG. 32 is a diagram illustrating an example of a case where motionestimation is further performed on a current block on a per-sub-blockbasis.

FIG. 33 is a diagram illustrating an example in which a reconstructedpixel region and a current block are partitioned on a per-sub-blockbasis.

FIG. 34 is a flowchart illustrating an example of an inter predictionmethod using a reconstructed pixel region.

FIG. 35 is a diagram illustrating an example in which reconstructedblocks neighboring a current block are used to partition a reconstructedpixel region into sub-blocks according to the present invention.

FIG. 36 is a diagram illustrating an example in which reconstructedblocks neighboring a current block are used to partition a current blockinto multiple sub-blocks according to the present invention.

FIG. 37 is a flowchart illustrating a method of partitioning a currentblock into multiple sub-blocks according to an embodiment of the presentinvention.

FIG. 38 is a flowchart illustrating a method of partitioning areconstructed region used to encode or decode a current block intomultiple sub-blocks according to an embodiment of the present invention.

FIG. 39 is a flowchart illustrating an example of an inter predictionmethod using the sub-blocks of the current block partitioned as shown inFIG. 36.

FIG. 40 is a flowchart illustrating a method of encoding informationdetermined according to inter prediction shown in FIG. 39.

FIG. 41 is a flowchart illustrating an example of a method of decodinginformation encoded by the encoding method shown in FIG. 40.

FIGS. 42a and 42b are diagrams illustrating a sixth exemplary embodimentof the present invention.

FIG. 43 is a flowchart illustrating an example of a method ofdetermining an inter prediction mode according to the sixth exemplaryembodiment of the present invention described with reference to FIGS.42a and 42 b.

FIG. 44 is a diagram illustrating a process of encoding informationdetermined by the method shown in FIG. 43.

FIG. 45 is a diagram illustrating a process of decoding informationencoded by the method shown in FIG. 44.

MODE FOR INVENTION

The present invention may be modified in various ways and implemented byvarious embodiments, so that specific embodiments are shown in thedrawings and will be described in detail. However, the present inventionis not limited thereto, and the exemplary embodiments can be construedas including all modifications, equivalents, or substitutes in atechnical concept and a technical scope of the present invention. Thesimilar reference numerals refer to the similar elements described inthe drawings.

Terms “first”, “second”, etc. can be used to describe various elements,but the elements are not to be construed as being limited to the terms.The terms are only used to differentiate one element from otherelements. For example, the “first” element may be named the “second”element without departing from the scope of the present invention, andsimilarly the “second” element may also be named the “first” element.The term “and/or” includes a combination of a plurality of items or anyone of a plurality of terms.

It will be understood that when an element is referred to as being“coupled” or “connected” to another element, it can be directly coupledor connected to the other element or intervening elements may be presenttherebetween. In contrast, it will be understood that when an element isreferred to as being “directly coupled” or “directly connected” toanother element, there are no intervening elements present.

The terms used in the present specification are merely used to describeparticular embodiments, and are not intended to limit the presentinvention. An expression used in the singular encompasses the expressionof the plural, unless it has a clearly different meaning in the context.In the present specification, it will be understood that terms such as“including”, “having”, etc. are intended to indicate the existence ofthe features, numbers, steps, actions, elements, parts, or combinationsthereof disclosed in the specification, and are not intended to precludethe possibility that one or more other features, numbers, steps,actions, elements, parts, or combinations thereof may exist or may beadded.

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings. Hereinafter, thesame elements in the drawings are denoted by the same referencenumerals, and a repeated description of the same elements will beomitted.

FIG. 1 is a block diagram illustrating an image encoding deviceaccording to an embodiment of the present invention.

Referring to FIG. 1, an image encoding device 100 may include an imagepartitioning module 101, an intra prediction module 102, an interprediction module 103, a subtractor 104, a transform module 105, aquantization module 106, an entropy encoding module 107, adequantization module 108, an inverse transform module 109, an adder110, a filter module 111, and a memory 112.

The constituents shown in FIG. 1 are independently shown so as torepresent different distinctive functions in the image encoding device,which does not mean that each constituent is constituted as separatedhardware or a single software constituent unit. In other words, eachconstituent includes each of enumerated constituents for convenience.Thus, at least two constituents of each constituent may be combined toform one constituent or one constituent may be divided into a pluralityof constituents to perform each function. The embodiment where eachconstituent is combined and the embodiment where one constituent isdivided are also included in the scope of the present invention, if notdeparting from the essence of the present invention.

Also, some of constituents may not be indispensable constituentsperforming essential functions of the present invention but be selectiveconstituents improving only performance thereof. The present inventionmay be implemented by including only the indispensable constituents forimplementing the essence of the present invention except theconstituents used in improving performance. The structure including onlythe indispensable constituents except the selective constituents used inimproving only performance is also included in the scope of the presentinvention.

The image partitioning module 101 may partition an input image into oneor more blocks. Here, the input image may have various shapes and sizes,such as a picture, a slice, a tile, a segment, and the like. A block maymean a coding unit (CU), a prediction unit (PU), or a transform unit(TU). The partitioning may be performed on the basis of at least oneamong a quadtree and a binary tree. Quadtree partitioning is a method ofpartitioning a parent block into four child blocks of which the widthand the height are half of those of the parent block. Binary treepartitioning is a method of partitioning a parent block into two childblocks of which either the width or the height is half of that of theparent block. Through the above-described partitioning based on binarytree, a block may be in a square shape as well as a non-square shape.

Hereinafter, in the embodiment of the present invention, the coding unitmay mean a unit of performing encoding or a unit of performing decoding.

The prediction modules 102 and 103 may include the intra predictionmodule 102 performing intra prediction and the inter prediction module103 performing inter prediction. Whether to perform inter prediction orintra prediction on the prediction unit may be determined, and detailedinformation (for example, an intra prediction mode, a motion vector, areference picture, and the like) depending on each prediction method maybe determined. Here, a processing unit on which prediction is performedmay be different from a processing unit in which the prediction methodand the detailed content are determined. For example, the predictionmethod, the prediction mode, and the like may be determined on aper-prediction unit basis, and prediction may be performed on aper-transform unit basis.

A residual value (residual block) between the generated prediction blockand an original block may be input to the transform module 105. Further,prediction mode information used for prediction, motion vectorinformation, and the like may be encoded with the residual value by theentropy encoding module 107 and then may be transmitted to a decoder.When a particular encoding mode is used, the original block is intactlyencoded and transmitted to the decoding module without generating aprediction block by the prediction module 102, 103.

The intra prediction module 102 may generate a prediction block on thebasis of information on a reference pixel around the current block,which is information on a pixel within a current picture. When theprediction mode of the neighboring block of the current block on whichintra prediction is to be performed is inter prediction, a referencepixel included in the neighboring block to which inter prediction hasbeen applied is replaced by a reference pixel within another neighboringblock to which intra prediction has been applied. That is, when thereference pixel is unavailable, at least one reference pixel amongavailable reference pixels is used instead of information on theunavailable reference pixel.

Prediction modes in intra prediction may include a directionalprediction mode using the information on the reference pixel dependingon a prediction direction and a non-directional prediction mode notusing directivity information in performing prediction. A mode forpredicting luma information may be different from a mode for predictingchroma information, and in order to predict the chroma information,intra prediction mode information used to predict the luma informationor predicted luma signal information may be utilized.

The intra prediction module 102 may include an adaptive intra smoothing(AIS) filter, a reference pixel interpolation module, and a DC filter.The AIS filter is a filter performing filtering on a reference pixel ofthe current block, and may adaptively determine whether to apply thefilter depending on a prediction mode of a current prediction unit. Whenthe prediction mode of the current block is a mode in which AISfiltering is not performed, the AIS filter is not applied.

When the intra prediction mode of the prediction unit is a predictionmode in which intra prediction is performed on the basis of a pixelvalue obtained by interpolating the reference pixel, the reference pixelinterpolation module of the intra prediction module 102 interpolates thereference pixel to generate a reference pixel at a position on aper-fraction basis. When the prediction mode of the current predictionunit is a prediction mode in which the prediction block is generatedwithout interpolating the reference pixel, the reference pixel is notinterpolated. The DC filter generates the prediction block throughfiltering when the prediction mode of the current block is a DC mode.

The inter prediction module 103 generates the prediction block using apre-reconstructed reference image stored in the memory 112 and motioninformation. The motion information may contain, for example, a motionvector, a reference picture index, a list 1 prediction flag, a list 0prediction flag, and the like.

In the image encoding device, there are two typical methods ofgenerating motion information.

The first method is a method in which motion information (a motionvector, a reference image index, an inter prediction direction, and thelike) is generated using a motion estimation process. FIG. 2 is adiagram illustrating a method of generating motion information by usingmotion estimation according to the conventional technology. Motionestimation is a method of generating motion information such as, amotion vector, a reference image index, and an inter predictiondirection of a current image region to be encoded using the decodedreference image after being encoded. It is possible that motionestimation is performed in the entire reference image or that in orderto reduce the complexity, a search range is set and motion estimation isperformed only within the search range.

The second method of generating the motion information is a method inwhich motion information of neighboring blocks of the current imageblock to be encoded is used.

FIG. 3 is a diagram illustrating an example of neighboring blocks thatmay be used to generate motion information of a current block. FIG. 3shows an example of spatial candidate blocks A to E and a temporalcandidate block COL as neighboring blocks that may be used to generatethe motion information of the current block. The spatial candidateblocks A to E are present in the same image as the current block, butthe temporal candidate block COL is present in an image that isdifferent from the image to which the current block belongs.

One piece of motion information of the spatial candidate blocks A to Eand the temporal candidate block COL, which neighbor the current block,may be selected as the motion information of the present block. Here, anindex may be defined that indicates which block has the motioninformation used as the motion information of the current block. Thisindex information also belongs to the motion information. In the imageencoding device, using the above methods, the motion information may begenerated and the prediction block may be generated through motioncompensation.

A residual block may be generated that includes residual valueinformation which is a difference value between the prediction unitgenerated by the prediction module 102, 103 and the original block ofthe prediction unit. The generated residual block may be input to thetransform module 105 for transform.

The inter prediction module 103 may derive the prediction block on thebasis of information on at least one picture among the previous pictureand the subsequent picture of the current picture. Further, theprediction block of the current block may be derived on the basis ofinformation on a partial region with encoding completed within thecurrent picture. The inter prediction module 103 according to anembodiment of the present invention may include a reference pictureinterpolation module, a motion prediction module, and a motioncompensation module.

The reference picture interpolation module may receive reference pictureinformation from the memory and may generate information on a pixelequal to or smaller than an integer pixel in the reference picture. Inthe case of a luma pixel, a DCT-based 8-tap interpolation filter havingdifferent filter coefficients may be used to generate information on apixel equal to or smaller than an integer pixel on a per-¼ pixel basis.In the case of a chroma signal, a DCT-based 4-tap interpolation filterhaving different filter coefficients may be used to generate informationon a pixel equal to or smaller than an integer pixel on a per-⅛ pixelbasis.

The motion prediction module may perform motion prediction on the basisof the reference picture interpolated by the reference pictureinterpolation module. As methods of calculating the motion vector,various methods, such as a full search-based block matching algorithm(FBMA), a three step search (TSS) algorithm, a new three-step search(NTS) algorithm, and the like, may be used. The motion vector may have amotion vector value on a per-½ or ¼ pixel basis on the basis of theinterpolated pixel. The motion prediction module may predict theprediction block of the current block by using different motionprediction methods. As motion prediction methods, various methods, suchas a skip method, a merge method, an advanced motion vector prediction(AMVP) method, and the like, may be used.

The subtractor 104 performs subtraction on the block to be currentlyencoded and on the prediction block generated by the intra predictionmodule 102 or the inter prediction module 103 so as to generate theresidual block of the current block.

The transform module 105 may transform the residual block containingresidual data, using a transform method, such as DCT, DST,Karhunen-Loeve transform (KLT), and the like. Here, the transform methodmay be determined on the basis of the intra prediction mode of theprediction unit that is used to generate the residual block. Forexample, depending on the intra prediction mode, DCT may be used in thehorizontal direction, and DST may be used in the vertical direction.

The quantization module 106 may quantize values transformed into afrequency domain by the transform module 105. Quantization coefficientsmay vary according to a block or importance of an image. The valuecalculated by the quantization module 106 may be provided to thedequantization module 108 and the entropy encoding module 107.

The transform module 105 and/or the quantization module 106 may beselectively included in the image encoding device 100. That is, theimage encoding device 100 may perform at least one of transform andquantization on residual data of the residual block, or may encode theresidual block by skipping both transform and quantization. Even thoughthe image encoding device 100 does not perform either transform orquantization or does not perform both transform and quantization, theblock that is input to the entropy encoding module 107 is generallyreferred to as a transform block. The entropy encoding module 107entropy encodes the input data. Entropy encoding may use variousencoding methods, for example, exponential Golomb coding,context-adaptive variable length coding (CAVLC), and context-adaptivebinary arithmetic coding (CABAC).

The entropy encoding module 107 may encode a variety of information,such as residual value coefficient information of a coding unit, blocktype information, prediction mode information, partitioning unitinformation, prediction unit information, transmission unit information,motion vector information, reference frame information, blockinterpolation information, filtering information, and the like, from theprediction module 102, 103. In the entropy encoding module 107, thecoefficient of the transform block may be encoded on a per-partial blockbasis within the transform block on the basis of various types of flagsindicating a non-zero coefficient, a coefficient of which the absolutevalue is higher than one or two, the sign of the coefficient, and thelike may be encoded. A coefficient that is not encoded only with theflags may be encoded through the absolute value of the differencebetween the coefficient encoded through the flag and the actualcoefficient of the transform block. The dequantization module 108dequantizes the values quantized by the quantization module 106, and theinverse transform module 109 inversely transforms the values transformedby the transform module 105. The residual value generated by thedequantization module 108 and the inverse transform module 109 may becombined with the prediction unit predicted through a motion estimationmodule included in the prediction module 102, 103, the motioncompensation module, and the intra prediction module 102 such that areconstructed block is generated. The adder 110 adds the predictionblock generated by the prediction module 102, 103 and the residual blockgenerated by the inverse transform module 109 so as to generate areconstructed block.

The filter module 111 may include at least one of a deblocking filter,an offset correction module, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion that occurs due toboundaries between the blocks in the reconstructed picture. In order todetermine whether to perform deblocking, whether to apply the deblockingfilter to the current block may be determined on the basis of the pixelsincluded in several rows and columns in the block. When the deblockingfilter is applied to the block, a strong filter or a weak filter isapplied depending on required deblocking filtering strength. Further, inapplying the deblocking filter, when performing horizontal directionfiltering and vertical direction filtering, horizontal directionfiltering and vertical direction filtering are configured to beprocessed in parallel.

The offset correction module may correct an offset from the originalimage on a per-pixel basis with respect to the image subjected todeblocking. In order to perform offset correction on a particularpicture, it is possible to use a method of separating pixels of theimage into the predetermined number of regions, determining a region tobe subjected to offset, and applying the offset to the determinedregion, or a method of applying an offset considering edge informationof each pixel.

Adaptive loop filtering (ALF) may be performed on the basis of the valueobtained by comparing the filtered reconstructed image and the originalimage. The pixels included in the image may be divided intopredetermined groups, a filter to be applied to each of the groups maybe determined, and filtering may be individually performed on eachgroup. Information on whether to apply ALF of a luma signal may betransmitted for each coding unit (CU), and the shape and the filtercoefficient of the ALF filter to be applied may vary depending on eachblock. Also, regardless of the characteristic of the application targetblock, the ALF filter in the same form (fixed form) may be applied.

The memory 112 may store the reconstructed block or picture calculatedthrough the filter module 111, and the stored reconstructed block orpicture may be provided to the prediction module 102, 103 whenperforming inter prediction.

Next, an image decoding device according to an embodiment of the presentinvention will be described with reference to the accompanying drawings.FIG. 4 is a block diagram illustrating an image decoding device 400according to an embodiment of the present invention.

Referring to FIG. 4, the image decoding device 400 may include anentropy decoding module 401, a dequantization module 402, an inversetransform module 403, an adder 404, a filter module 405, a memory 406,and prediction modules 407 and 408.

When an image bitstream generated by the image encoding device 100 isinput to the image decoding device 400, the input bitstream is decodedaccording to a reverse process of the process performed in the imageencoding device 100.

The entropy decoding module 401 may perform entropy decoding accordingto the reverse procedure of the entropy encoding performed by theentropy encoding module 107 of the image encoding device 100. Forexample, corresponding to the methods performed by the image encoder,various methods, such as exponential Golomb coding, context-adaptivevariable length coding (CAVLC), and context-adaptive binary arithmeticcoding (CABAC), may be applied. In the entropy decoding module 401, thecoefficient of the transform block may be encoded on a per-partial blockbasis within the transform block on the basis of various types of flagsindicating a non-zero coefficient, a coefficient of which the absolutevalue is higher than one or two, the sign of the coefficient, and thelike. A coefficient that is not represented only by the flags may bedecoded through the sum of a coefficient represented through the flagand a signaled coefficient.

The entropy decoding module 401 may decode information related to intraprediction and inter prediction performed in the encoder. Thedequantization module 402 performs dequantization on the quantizedtransform block to generate the transform block. This operatessubstantially in the same manner as the dequantization module 108 inFIG. 1.

The inverse transform module 403 performs inverse transform on thetransform block to generate the residual block. Here, the transformmethod may be determined on the basis of the prediction method (inter orintra prediction), the size and/or the shape of the block, informationon the intra prediction mode, and the like. This operates substantiallyin the same manner as the inverse transform module 109 in FIG. 1.

The adder 404 adds the prediction block generated by the intraprediction module 407 or the inter prediction module 408 and theresidual block generated by the inverse transform module 403 so as togenerate a reconstructed block. This operates substantially in the samemanner as the adder 110 in FIG. 1.

The filter module 405 reduces various types of noises occurring in thereconstructed blocks.

The filter module 405 may include a deblocking filter, an offsetcorrection module, and an ALF.

From the image encoding device 100, it is possible to receiveinformation on whether or not the deblocking filter is applied to theblock or picture and information on whether the strong filter is appliedor the weak filter is applied when the deblocking filter is applied. Thedeblocking filter of the image decoding device 400 may receiveinformation related to the deblocking filter from the image encodingdevice 100, and the image decoding device 400 may perform deblockingfiltering on the block.

The offset correction module may perform offset correction on thereconstructed image on the basis of the type of offset correction,offset value information, and the like applied to the image inperforming encoding.

The ALF may be applied to the coding unit on the basis of information onwhether to apply the ALF, ALF coefficient information, and the likereceived from the image encoding device 100. The ALF information may beprovided by being included in a particular parameter set. The filtermodule 405 operates substantially in the same manner as the filtermodule 111 in FIG. 1.

The memory 406 stores the reconstructed block generated by the adder404. This operates substantially in the same manner as the memory 112 inFIG. 1.

The prediction module 407, 408 may generate a prediction block on thebasis of information related to prediction block generated provided fromthe entropy decoding module 401 and of information on the previouslydecoded block or picture provided from the memory 406.

The prediction modules 407 and 408 may include the intra predictionmodule 407 and the inter prediction module 408. Although not shown, theprediction module 407, 408 may further include a prediction unitdetermination module. The prediction unit determination module mayreceive a variety of information input from the entropy decoding module401, such as prediction unit information, prediction mode information ofan intra prediction method, information related to motion prediction ofan inter prediction method, and the like, may separate a prediction unitin a current coding unit, and may determine whether inter prediction isperformed on the prediction unit or intra prediction is performed on theprediction unit. By using information required in inter prediction ofthe current prediction unit provided from the image encoding device 100,the inter prediction module 408 may perform inter prediction on thecurrent prediction unit on the basis of information included at leastone picture among the previous picture and the subsequent picture of thecurrent picture including the current prediction unit. Alternatively,inter prediction may be performed on the basis of information on somepre-reconstructed regions within the current picture including thecurrent prediction unit.

In order to perform inter prediction, on the basis of the coding unit,it may be determined which mode among a skip mode, a merge mode, and anAMVP Mode is used for the motion prediction method of the predictionunit included in the coding unit.

The intra prediction module 407 generates the prediction block using thepre-reconstructed pixels positioned near the block to be currentlyencoded.

The intra prediction module 407 may include an adaptive intra smoothing(AIS) filter, a reference pixel interpolation module, and a DC filter.The AIS filter is a filter performing filtering on the reference pixelof the current block, and may adaptively determine whether to apply thefilter depending on the prediction mode of the current prediction unit.The prediction mode of the prediction unit provided from the imageencoding device 100 and the AIS filter information may be used toperform AIS filtering on the reference pixel of the current block. Whenthe prediction mode of the current block is a mode in which AISfiltering is not performed, the AIS filter is not applied.

When the prediction mode of the prediction unit is a prediction mode inwhich intra prediction is performed on the basis of a pixel valueobtained by interpolating the reference pixel, the reference pixelinterpolation module of the intra prediction module 407 interpolates thereference pixel to generate a reference pixel at a position on aper-fraction basis. The generated reference pixel on a per-fractionbasis may be used as a prediction pixel of a pixel within the currentblock. When the prediction mode of the current prediction unit is aprediction mode in which a prediction block is generated withoutinterpolating the reference pixel, the reference pixel is notinterpolated. The DC filter may generate a prediction block throughfiltering when the prediction mode of the current block is a DC mode.

The intra prediction module 407 operates substantially in the samemanner as the intra prediction module 102 in FIG. 1.

The inter prediction module 408 generates an inter prediction blockusing a reference picture stored in the memory 406 and motioninformation. The inter prediction module 408 operates substantially inthe same manner as the inter prediction module 103 in FIG. 1.

Hereinafter, various embodiments of the present invention will bedescribed in detail with reference to the drawings.

First Exemplary Embodiment

FIGS. 5a and 5b are diagrams illustrating inter prediction using areconstructed pixel region according to a first exemplary embodiment ofthe present invention. In the inter prediction using the reconstructedpixel region according to the embodiment, particularly, the motionvector of the current block may be derived using the reconstructed pixelregion.

FIG. 5a shows a current block 51 to be encoded or decoded and apre-reconstructed pixel region C 52 as a region adjacent to the currentblock 51. The current block 51 and the reconstructed pixel region C 52are included in the current image 50. The current image 50 may be apicture, a slice, a tile, a coding tree block, a coding block, or otherimage regions. The reconstructed pixel region C 52 may correspond to areconstructed pixel region after being encoded before encoding of thecurrent block 51 in terms of encoding, and may be a pre-reconstructedregion before decoding of the current block 51 in terms of decoding.

Before encoding or decoding of the current block, the reconstructedpixel region C 52 neighbors the current block 51, and thus the imageencoding device 100 and the image decoding device 400 may use the samereconstructed pixel region C 52. Therefore, without encoding the motioninformation of the current block 51 by the image encoding device 100,the reconstructed pixel region C 52 is used such that the image encodingdevice 100 and the image decoding device 400 may generate the motioninformation of the current block 51 and the prediction block in the samemanner.

FIG. 5b shows an example of motion estimation and motion compensationusing a reconstructed pixel region. A reference image 53 shown in FIG.5b is searched for a region matched with the reconstructed pixel regionC 52 shown in FIG. 5a . When a reconstructed pixel region D 54 that ismost similar to the reconstructed pixel region C 52 is determined, adisplacement between a region 56, which is at the same position as thereconstructed pixel region C 52, and the reconstructed pixel region D 54is determined to be a motion vector 57 of the reconstructed pixel regionC 52. The motion vector 57 determined as described above is selected asthe motion vector of the current block 51, and a prediction block 58 ofthe current block 51 may be derived using the motion vector 57.

FIG. 6 is a flowchart illustrating an inter prediction method accordingto the first exemplary embodiment of the present invention.

Inter prediction according to the embodiment may be performed by theinter prediction module 103 of the image encoding device 100 or by theinter prediction module 408 of the image decoding device 400. Referenceimages used in inter prediction are stored in the memory 112 of theimage encoding device 100 or in the memory 406 of the image decodingdevice 400. The inter prediction module 103 or the inter predictionmodule 408 may generate the prediction block of the current block 51with reference to the reference image stored in the memory 112 or in thememory 406.

Referring to FIG. 6, first, the reconstructed pixel region 52 isselected to be used in deriving of the motion vector of the currentblock to be encoded or decoded, at step S61. Next, on the basis of thereconstructed pixel region 52 and the reference image of the currentblock, the motion vector of the reconstructed pixel region 52 is derivedat step S63. As described above with reference to FIG. 5b , thereference image 53 shown in FIG. 5b is searched for the region matchedwith the reconstructed pixel region C 52. When the reconstructed pixelregion D 54 most similar to the reconstructed pixel region C 52 isdetermined, the displacement between the region 56 at the same positionas the reconstructed pixel region C 52 and the reconstructed pixelregion D 54 is determined to be the motion vector 57 of thereconstructed pixel region C 52.

The image encoding device 100 or the image decoding device 400 selectsthe motion vector 57 of the reconstructed pixel region C 52, determinedas described above, as the motion vector of the current block 51 at stepS65. Using this motion vector 57, the prediction block 58 of the currentblock 51 may be generated.

In the meantime, the reconstructed pixel region C 52 may be in variousshapes and/or sizes. FIGS. 7a to 7c are diagrams illustrating examplesof reconstructed pixel regions. The letters M, N, O, and P shown inFIGS. 7a to 7c denotes pixel intervals, respectively, and it is possiblethat O and P have negative values assuming that the absolute values of Oand P are lower than the horizontal or vertical lengths of the currentblock, respectively.

Also, it is possible that the reconstructed pixel regions at the upperside and the left side of the current block are used as thereconstructed pixel region C 52 or that the two regions are combinedinto a single piece to be used as the reconstructed pixel region C 52.Also, it is possible that the reconstructed pixel region C 52 is used bybeing subjected to subsampling. In this method, only the decodedinformation around the current block is used to derive the motioninformation, and thus it is not necessary to transmit the motioninformation from the encoding device 100 to the decoding device 400.

According to the embodiment of the present invention, the decodingdevice 400 also performs motion estimation, so that if motion estimationis performed on the entire reference image, the complexity may extremelyincrease. Therefore, by transmitting the search range on a per-blockbasis or in the parent header or by fixing the search region to be thesame in the encoding device 100 and in the decoding device 400,computational complexity of the decoding device 400 may be reduced.

FIG. 8 is a flowchart illustrating a process of determining an interprediction method according to an embodiment of the present invention.Among the inter prediction using the reconstructed pixel regionaccording to the present invention and the conventional inter predictionmethod, the optimum method may be determined through rate-distortionoptimization (RDO). The process shown in FIG. 8 may be performed by theimage encoding device 100.

Referring to FIG. 8, first, inter prediction according to theconventional method is performed to compute cost_A at step S81, and thenas described above, inter prediction using the reconstructed pixelregion according to the present invention is performed to compute cost_Bat step S82.

Afterward, cost_A is compared with cost_B to determine which method isoptimum to use, at step S83. When cost_A is lower, it is set to performinter prediction using the conventional method at step S84. Otherwise,it is set to perform inter prediction using the reconstructed pixelregion at step S85.

FIG. 9 is a diagram illustrating a process of encoding information thatindicates an inter prediction method determined by the process shown inFIG. 8. Hereinafter, the information indicating the inter predictionmethod determined by the process shown in FIG. 8 is referred to asdecoder-side motion vector derivation (DMVD) indication information. TheDMVD indication information or decoder-side motion vector derivationindication information may be information indicating whether interprediction using the conventional method is performed or interprediction using the reconstructed pixel region is performed.

Referring to FIG. 9, the DMVD indication information indicating theinter prediction method determined by the process shown in FIG. 8 isencoded at step S91. The DMVD indication information may be, forexample, a 1-bit flag or one of several indexes. Afterward, the motioninformation is encoded at step S92, and the algorithm ends.

Alternatively, information indicating whether or not inter predictionusing the reconstructed pixel region according to the embodiment of thepresent invention is used may be generated in the parent header firstand then may be decoded. That is, when the information indicatingwhether or not inter prediction using the reconstructed pixel region isused indicates true, the DMVD indication information is encoded. Whenthe information indicating whether or not inter prediction using thereconstructed pixel region is used indicates false, the DMVD indicationinformation is not present within the bitstream and in this case, thecurrent block is predicted using the conventional inter prediction.

In the meantime, regarding the parent header, the parent headerincluding the information that indicates whether or not inter predictionusing the reconstructed pixel region is used may be transmitted by beingincluded in a block header, a slice header, a tile header, a pictureheader, or a sequence header.

FIG. 10 is a diagram illustrating a process of decoding the DMVDindication information encoded as shown in FIG. 9.

The decoding device 400 decodes the DMVD indication information at stepS101, decodes the motion information at step S102, and ends thealgorithm.

In the case where the information indicating whether or not interprediction using the reconstructed pixel region is used is present inthe parent header of the bitstream, when the information indicatingwhether or not inter prediction using the reconstructed pixel region isused indicates true, the DMVD indication information is present in thebitstream. When the information indicating whether or not interprediction using the reconstructed pixel region is used indicates false,the DMVD indication information is not present within the bitstream andin this case, the current block is predicted using the conventionalinter prediction.

Regarding the parent header, the parent header including the informationthat indicates whether or not inter prediction using the reconstructedpixel region is used may be transmitted by being included in a blockheader, a slice header, a tile header, a picture header, or a sequenceheader.

Second Exemplary Embodiment

Hereinafter, the second exemplary embodiment of the present inventionwill be described with reference to the drawings.

In the second exemplary embodiment, the inter prediction using thereconstructed pixel region according to the first exemplary embodimentdescribed above is applied to inter prediction using affinetransformation. Specifically, in order to derive a motion vector of acontrol point used for inter prediction using affine transformation, amotion vector derivation method using the reconstructed pixel region isapplied. Hereinafter, for convenience of description, according to thesecond exemplary embodiment of the present invention, inter predictionusing affine transformation is simply referred to as affine interprediction.

FIG. 11 is a diagram illustrating affine inter prediction.

In affine inter prediction, motion vectors at four corners of thecurrent block to be encoded or decoded are obtained, and then the motionvectors are used to generate a prediction block. Here, the four cornersof the current block may correspond to the control points.

Referring to FIG. 11, a block identified by motion vectors 11-2, 11-3,11-4, and 11-5 at the four corners (namely, the control points) of thecurrent block (not shown) within the current image may be a predictionblock 11-6 of the current block.

This affine inter prediction enables prediction of a block or imageregion subjected to rotation, zoom-in/zoom-out, translation, reflection,or shear deformation.

Equation 1 below is a general determinant of affine transformation.

$\begin{matrix}{\begin{bmatrix}x^{\prime} \\y^{\prime} \\1\end{bmatrix} = {\begin{bmatrix}a & b & e \\c & d & f \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}x \\y \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Equation 1 is an equation representing transform of two-dimensionalcoordinates, wherein (x, y) denotes original coordinates, (x′, y′)denotes destination coordinates, and a, b, c, d, e, and f denotetransform parameters.

In order to apply this affine transformation to video codec, transformparameters need to be transmitted to the image decoding device, whichresults in enormous increase in overhead. For this reason, in theconventional video codec, affine transformation is simply applied usingN reconstructed neighboring control points.

Equation 2 below represents a method of deriving a motion vector of anarbitrary sub-block within the current block by using two control pointsat the top left and the top right of the current block.

                                 [Equation  2]$\left\{ \begin{matrix}{{MV}_{x} = {{\frac{\left( {{MV}_{1x} - {MV}_{0x}} \right.}{W}x} - {\frac{\left( {{MV}_{1y} - {MV}_{0y}} \right)}{W}y} + {MV}_{0x}}} \\{{MV}_{y} = {{\frac{\left( {{MV}_{1y} - {MV}_{0y}} \right.}{W}x} + {\frac{\left( {{MV}_{1x} - {MV}_{0x}} \right)}{W}y} + {MV}_{0y}}}\end{matrix} \right.$

In Equation 2, (x, y) denotes the position of the arbitrary sub-blockwithin the current block, W denotes the horizontal length of the currentblock, (MV_(x), MV_(y)) denotes the motion vector of the sub-block,(MV_(0x), MV_(0y)) denotes the motion vector of the top left controlpoint, and (MV_(1x), MV_(1y)) denotes the motion vector of the top rightcontrol point.

Next, Equation 3 below represents a method of deriving a motion vectorof an arbitrary sub-block within the current block by using threecontrol points at the top left, the top right, and the bottom left ofthe current block.

$\quad\begin{matrix}\left\{ \begin{matrix}{{MV}_{x} = {{\frac{\left( {{MV}_{1x} - {MV}_{0x}} \right.}{W}x} + {\frac{\left( {{MV}_{2x} - {MV}_{0x}} \right)}{H}y} + {MV}_{0x}}} \\{{MV}_{y} = {{\frac{\left( {{MV}_{1y} - {MV}_{0y}} \right.}{W}x} + {\frac{\left( {{MV}_{2y} - {MV}_{0y}} \right)}{H}y} + {MV}_{0y}}}\end{matrix} \right. & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

In Equation 3, (x, y) denotes the position of the arbitrary sub-block, Wand H denote the horizontal length and the vertical length of thecurrent block, respectively, (MV_(x), MV_(y)) denotes the motion vectorof the sub-block within the current block, (MV_(0x), MV_(0y)) denotesthe motion vector of the top left control point, (MV_(1x), MV_(1y))denotes the motion vector of the top right control point, and (MV_(2x),MV_(2y)) denotes the motion vector of the bottom left control point.

In the second exemplary embodiment of the present invention, in order toderive the motion vector of the control point used for affine interprediction, the motion vector derivation method using the reconstructedpixel region according to the first exemplary embodiment is applied.Therefore, the image encoding device 100 does not need to transmitmotion vector information of multiple control points to the imagedecoding device 400.

FIGS. 12a and 12b are diagrams illustrating derivation of the motionvector of the control point by using the reconstructed pixel regionaccording to the second exemplary embodiment of the present invention.

Referring to FIG. 12a , a current block 12 a-2 to be encoded or decodedis included in a current image 12 a-1. Four control points for affineinter prediction are denoted by circles at four corners of the currentblock 12 a-2 in FIG. 12a . Also, in this figure, as regions adjacent tothree control points at the top left, the top right, and the bottomleft, pre-reconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5are shown.

In the embodiment, motion vectors of three control points at the topleft, the top right, and the bottom left are derived using thereconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5 as shown inFIG. 12b . However, in the case of the control point at the bottom rightof the current block, a reconstructed pixel region may be not presentnearby. In this case, by using a sub-block d 12 a-6 in an arbitrarysize, the motion vector of the sub-block d 12 a-6 obtained using theconventional inter prediction method may be set to be the motion vectorof the bottom right control point of the current block.

Referring to FIG. 12b , a reference image 12 b-1 shown in FIG. 12b issearched for regions d 12 b-10, e 12 b-11, and f 12 b-12 matched withthe reconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5,respectively, shown in FIG. 12a . The displacements from the regions 12b-6, 12 b-7, and 12 b-8, which are at the same position as thereconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5,respectively, are determined to be motion vectors 12 b-2, 12 b-3, and 12b-4 of the reconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5,respectively. The motion vectors 12 b-2, 12 b-3, and 12 b-4 determinedas described above are determined to be the motion vectors of threecontrol points at the top left, the top right, and the bottom left ofthe current block 12 a-2. In the meantime, as the motion vector of thecontrol point at the bottom right, the motion vector of the sub-block d12 a-6 obtained using the conventional inter prediction method may beused.

By using the motion vectors of the four control points derived asdescribed above, a motion vector of an arbitrary sub-block within thecurrent block may be derived as shown in Equation 4 below.

                                 [Equation  4]$\left\{ \begin{matrix}{{MV}_{x} = {{\frac{\frac{\begin{matrix}{\left( {{MV}_{1x} - {MV}_{0x}} \right) +} \\\left( {{MV}_{3x} - {MV}_{2x}} \right)\end{matrix}}{2}}{W\;}x} + {\frac{\frac{\begin{matrix}{\left( {{MV}_{2x} - {MV}_{0x}} \right) +} \\\left( {{MV}_{3x} - {MV}_{1x}} \right)\end{matrix}}{2}}{H}y} + {MV}_{0x}}} \\{{MV}_{y} = {{\frac{\frac{\begin{matrix}{\left( {{MV}_{1y} - {MV}_{0y}} \right) +} \\\left( {{MV}_{3y} - {MV}_{2y}} \right)\end{matrix}}{2}}{W}x} + {\frac{\frac{\begin{matrix}{\left( {{MV}_{2y} - {MV}_{0y}} \right) +} \\\left( {{MV}_{3y} - {MV}_{1y}} \right)\end{matrix}}{2}}{H}y} + {MV}_{0y}}}\end{matrix} \right.$

In Equation 4, (x, y) denotes the position of the arbitrary sub-blockwithin the current block, W and H denote the horizontal length and thevertical length of the current block, respectively, (MV_(x), MV_(y))denotes the motion vector of the sub-block within the current block,(MV_(0x), MV_(0y)) denotes the motion vector of the top left controlpoint, (MV_(1x), MV_(1y)) denotes the motion vector of the top rightcontrol point, (MV_(2x), MV_(2y)) denotes the motion vector of thebottom left control point, and (MV_(3x), MV_(3y)) denotes the motionvector of the bottom right control point.

In the meantime, the reconstructed pixel regions a 12 a-3, b 12 a-4, andc 12 a-5 may be in various sizes and/or shapes as described above withreference to FIGS. 7a to 7c . The size and/or the shape of the sub-blockd 12 a-6 may be the same as a preset size and/or a preset shape in theencoding device 100 and the decoding device 400. Also, on a per-blockbasis or through the parent header, horizontal and/or vertical sizeinformation of the sub-block d 12 a-6 may be transmitted, or sizeinformation may be transmitted on a per-exponentiation of two basis.

As described above, when motion vectors are derived from four controlpoints, these vectors are used to derive the motion vector of thecurrent block 12 a-2 or the motion vector of an arbitrary sub-blockwithin the current block 12 a-2, and this derived motion vector may beused to derive the prediction block of the current block 12 a-2 or theprediction block of an arbitrary sub-block within the current block 12a-2. Specifically, referring to Equation 4 above, the position of thecurrent block 12 a-2 is coordinates (0, 0), so that the motion vector ofthe current block 12 a-2 is the motion vector (MV_(0x), MV_(0y)) of thetop left control point. Therefore, the prediction block of the currentblock 12 a-2 may be obtained using the motion vector of the top leftcontrol point. When the current block is a 8×8 block and is partitionedinto four 4×4 sub-blocks, the motion vector of the sub-block at theposition (3,0) within the current block is obtained by substituting avalue of three for the variable x in Equation 4 above, a value of zerofor the variable y, and a value of eight for both variables W and H.

Next, with reference to FIG. 13, the inter prediction method accordingto the second exemplary embodiment of the present invention will bedescribed. FIG. 13 is a flowchart illustrating the inter predictionmethod according to the second exemplary embodiment of the presentinvention.

Inter prediction according to the embodiment may be performed by theinter prediction module 103 of the image encoding device 100 or theinter prediction module 408 of the image decoding device 400. Referenceimages used in inter prediction are stored in the memory 112 of theimage encoding device 100 or in the memory 406 of the image decodingdevice 400. The inter prediction module 103 or the inter predictionmodule 408 may generate the prediction block of the current block 51with reference to a reference image stored in the memory 112 or thememory 406.

Referring to FIG. 13, first, at least one reconstructed pixel region isselected to be used in deriving a motion vector of at least one controlpoint of the current block to be encoded or decoded, at step S131. Inthe embodiment shown in FIGS. 12a and 12b , to derive motion vectors ofthree control points at the top left, the top right, and the bottom leftof the current block 12 a-2, three reconstructed pixel regions a 12 a-3,b 12 a-4, and c 12 a-5 are selected. However, without being limitedthereto, to derive a motion vector of one or two control points amongthe three control points, one or two reconstructed pixel regions may beselected.

Next, on the basis of the at least one reconstructed pixel regionselected at step S131 and the reference image of the current block, amotion vector of at least one reconstructed pixel region is derived atstep S133. The image encoding device 100 or the image decoding device400 selects each motion vector of the reconstructed pixel region C 52,determined as described above, as a motion vector of at least onecontrol point of the current block at step S135. At least one motionvector selected as described above may be used to generate theprediction block of the current block.

FIG. 14 is a flowchart illustrating a process of determining an interprediction method according to the second exemplary embodiment of thepresent invention. According to the second exemplary embodiment of thepresent invention, among the affine inter prediction and theconventional inter prediction, the optimum method may be determinedthrough rate-distortion optimization (RDO). The process shown in FIG. 14may be performed by the image encoding device 100.

Referring to FIG. 14, first, inter prediction is performed using theconventional method to compute cost_A at step S141, and as describedabove, according to the second exemplary embodiment of the presentinvention, affine inter prediction is performed to compute cost_B atstep S142.

Afterward, cost_A is compared with cost_B to determine which method isoptimum to use, at step S143. When cost_A is lower, it is set to performinter prediction using the conventional method at step S144. Otherwise,it is set to perform affine inter prediction at step S145 according tothe second exemplary embodiment of the present invention.

FIG. 15 is a diagram illustrating a process of encoding information thatindicates an inter prediction method determined by the process shown inFIG. 14. Hereinafter, the information indicating the inter predictionmethod determined by the process shown in FIG. 14 is referred to asdecoder-side control point motion vector derivation (DCMVD) indicationinformation. The DCMVD indication information or decoder-side controlpoint motion vector derivation indication information may be informationindicating whether inter prediction using the conventional method isperformed or affine inter prediction according to the second exemplaryembodiment of the present invention is performed.

Referring to FIG. 15, the DCMVD indication information indicating theinter prediction method determined by the process shown in FIG. 14 isencoded at step S151. The DCMVD indication information may be, forexample, a 1-bit flag or one of several indexes. Afterward, the motioninformation is encoded at step S152, and the algorithm ends.

In the meantime, according to the second exemplary embodiment of thepresent invention, the information indicating whether or not affineinter prediction is used may be generated in the parent header first andthen may be encoded. That is, according to the second exemplaryembodiment of the present invention, when the information indicatingwhether or not affine inter prediction is used indicates true, the DCMVDindication information is encoded. According to the second exemplaryembodiment of the present invention, when the information indicatingwhether or not affine inter prediction is used indicates false, theDCMVD indication information is not present within the bitstream, and inthis case, the current block is predicted using the conventional interprediction.

In the meantime, regarding the parent header, the parent headerincluding the information indicating whether or not affine interprediction according to the present invention is used may be transmittedby being included in a block header, a slice header, a tile header, apicture header, or a sequence header.

FIG. 16 is a diagram illustrating a process of decoding the DCMVDindication information encoded as shown in FIG. 15.

The decoding device 400 decodes the DCMVD indication information at stepS161, decodes the motion information at step S162, and ends thealgorithm.

In the case where the information indicating whether or not affine interprediction according to the second exemplary embodiment of the presentinvention is used is present in the parent header of the bitstream, whenthe information indicating whether or not inter prediction using thereconstructed pixel region is used indicates true, the DCMVD indicationinformation is present in the bitstream. According to the secondexemplary embodiment of the present invention, when the informationindicating whether or not affine inter prediction is used indicatesfalse, the DCMVD indication information is not present within thebitstream, and in this case, the current block is predicted using theconventional inter prediction.

According to the second exemplary embodiment of the present invention,regarding the parent header, the parent header including the informationindicating whether or not affine inter prediction is used may betransmitted by being included in a block header, a slice header, a tileheader, a picture header, or a sequence header.

FIG. 17 is a flowchart illustrating an example of an image decodingmethod in which motion vectors of three control points are derived usinga reconstructed pixel region so as to generate a prediction block of acurrent block. The process shown in FIG. 17 relates to the embodimentshown in FIGS. 12a and 12 b.

To derive motion vectors of three control points at the top left, thetop right, and the bottom left of the current block 12 a-2, threereconstructed pixel regions a 12 a-3, b 12 a-4, and c 12 a-5 areselected. However, without being limited thereto, to derive a motionvector of one or two control points among the three control points, oneor two reconstructed pixel regions may be selected.

The image decoding device 400 may determine, on the basis of the DCMVDindication information, which inter prediction is to be performed. Whenthe DCMVD indication information indicates use of affine interprediction according to the present invention at step S171, the motionvectors of the control points at the top left, the top right, and thebottom left of the current block are estimated and selected using therespective reconstructed pixel regions at step S172.

Afterward, the motion vector obtained by decoding the transmitted motioninformation in the bitstream is set to be the motion vector of thecontrol point at the bottom right at step S173. Using affinetransformation in which the motion vectors of the four control pointsderived through steps S172 and S173 are used, an inter prediction blockof the current block is generated at step S174. When affine interprediction is not used, the prediction block of the current block isgenerated at step S175 according to the conventional inter prediction inwhich the motion information is decoded and the decoded motioninformation is used.

Third Exemplary Embodiment

FIG. 18 is a diagram illustrating a current block partitioned intomultiple regions for inter prediction according to a third exemplaryembodiment of the present invention.

FIG. 18 shows a current block 500 to be encoded or decoded and apre-reconstructed pixel region C 503 as a region adjacent to the currentblock 500. The current block 500 is partitioned into region A 500-a andregion B 500-b.

Due to correlation between pixels, the pixels within the reconstructedpixel region C 503 is likely to be similar to the pixels included in theregion A 500-a, but is unlikely to be similar to the pixels included inthe region B 500-b. Therefore, in inter prediction on the region A500-a, motion estimation and motion compensation using the reconstructedpixel region C 503 are performed to find accurate motion whilepreventing increase in overhead. In the meantime, as the interprediction method for the region B 500-b, the conventional interprediction may be applied.

FIG. 19 is a flowchart illustrating an inter prediction method accordingto a third exemplary embodiment of the present invention.

Inter prediction according to the embodiment may be performed by theinter prediction module 103 of the image encoding device 100 or by theinter prediction module 408 of the image decoding device 400. Thereference images used in inter prediction are stored in the memory 112of the image encoding device 100 or the memory 406 of the image decodingdevice 400. The inter prediction module 103 or the inter predictionmodule 408 may generate, with reference to the reference image stored inthe memory 112 or the memory 406, the prediction block of the region A500-a and the prediction block of the region B 500-b within the currentblock.

First, as shown in FIG. 18, the current block to be encoded or decodedis partitioned into multiple regions including a first region and asecond region at step S51. Here, the first region and the second regionmay correspond to the region A 500-a and the region B 500-b shown inFIG. 5, respectively. The current block 500 shown in FIG. 18 ispartitioned into two regions, the region A 500-a and the region B 500-b,but may be partitioned into three or more regions and may be partitionedinto regions in various sizes and/or shapes.

Next, using different inter prediction methods, a prediction block ofthe first region and a prediction block of the second region areobtained at step S53. Here, the inter prediction method for the region A500-a may be, as described above, the method in which motion estimationand motion compensation using the reconstructed pixel region C 503 areperformed. As the inter prediction method for the region B 500-b, theconventional inter prediction may be applied.

As in the embodiment, a method in which the current block is partitionedinto multiple regions and the prediction blocks of the respectiveregions are derived using different inter prediction methods is referredto as a mixed inter prediction.

FIG. 20 is a diagram illustrating an example of motion estimation andmotion compensation using a reconstructed pixel region.

Referring to FIG. 20, a reference image 600 is searched for a regionmatched with the reconstructed pixel region C 503 shown in FIG. 18. Asshown in FIG. 20, when a reconstructed pixel region D 603 that is mostsimilar to the reconstructed pixel region C 503 is determined, adisplacement between a region 601, which is at the same position as thereconstructed pixel region C 503, and the reconstructed pixel region D603 is selected as the motion vector 605 of the region A 500-a.

That is, the motion vector 605 estimated using the reconstructed pixelregion C 503 is selected as the motion vector of the region A 500-a ofthe current block. Using the motion vector 605, the prediction block ofthe region A 500-a is generated.

In the meantime, as shown in FIGS. 7a to 7c , the reconstructed pixelregion C 503 may be in various shapes and/or sizes. Also, it is possiblethat the upper and left sides of the reconstructed pixel region areseparated for use. Also, it is possible that the reconstructed pixelregion is used by being subjected to subsampling. In this method, onlythe decoded information around the current block is used to derive themotion information, and thus it is not necessary to transmit the motioninformation from the encoding device 100 to the decoding device 400.

According to the embodiment of the present invention, the decodingdevice 400 also performs motion estimation, so that if motion estimationis performed on the entire reference image, the complexity may extremelyincrease. Therefore, by transmitting the search range on a per-blockbasis or in the parent header or by fixing the search range to be thesame in the encoding device 100 and in the decoding device 400,computational complexity of the decoding device 400 may be reduced.

In the meantime, when estimating and encoding the motion vector of theregion B 500-b shown in FIG. 18, the motion information of the decodedblock within the reconstructed pixel region C 503 is used to predict themotion vector of the region B 500-b and the residual vectorcorresponding to the difference between the motion vector of the regionB 500-b and the prediction motion vector is encoded.

Alternatively, it is possible that the motion vector 605 estimated asthe motion vector of the region A 500-a is used to predict the motionvector of the region B 500-b and the residual vector is encoded.

Alternatively, it is possible that the motion vector of the decodedblock within the reconstructed pixel region C 503 and the estimatedmotion vector 605 of the region A 500-a are used to constitute a motionvector prediction set, the motion vector of the region B 500-b ispredicted, and the residual vector is encoded.

Alternatively, it is possible that among the blocks adjacent to thecurrent block, motion information is taken from a preset position toperform block merging. Here, block merging means that neighboring motioninformation is intactly applied to a block to be encoded. Here, it isalso possible that after setting several preset positions, an indexindicating at which position block merging is performed is used.

Further, possibly, the size of the region B 500-b is encoded by theencoding device 100 to be transmitted to the decoder on a per-blockbasis or through the parent header, or uses the same preset value orratio in the encoding device 100 and the decoding device 400.

FIG. 21 is a flowchart illustrating a process of determining an interprediction method according to an embodiment of the present invention.Among the mixed inter prediction according to the present invention andthe conventional inter prediction method, the optimum method may bedetermined through rate-distortion optimization (RDO). The process shownin FIG. 21 may be performed by the image encoding device 100.

Referring to FIG. 21, first, inter prediction is performed using theconventional method to compute cost_A at step S801, and as describedabove, the mixed inter prediction according to the present invention,the current block is partitioned into at least two regions to besubjected to inter prediction individually, is performed to computecost_B at step S802.

Afterward, cost_A and cost_B are computed to determine which method isoptimum to use, at step S803. When cost_A is lower, it is set to performthe inter prediction using the conventional method at step S804.Otherwise, it is set to perform the mixed inter prediction at step S805.

FIG. 22 is a diagram illustrating a process of transmitting, to thedecoding device 400, information that indicates the inter predictionmethod determined by the process shown in FIG. 21.

Information indicating which type of inter prediction has been used forthe block to be currently encoded is encoded at step S901. Thisinformation may be, for example, a 1-bit flag or one of several indexes.Afterward, the motion information is encoded at step S902, and thealgorithm ends.

Alternatively, the information indicating whether or not the mixed interprediction according to the embodiment of the present invention is usedmay be generated in the parent header first and then may be encoded.That is, when in the parent header, the information indicating whetheror not the mixed inter prediction is used indicates true, theinformation indicating which type of inter prediction has been used forthe block to be currently encoded is encoded. When the informationindicating whether or not the mixed inter prediction is used indicatesfalse, the information indicating which type of inter prediction hasbeen used is not present within the bitstream, and in this case, thecurrent block is not partitioned into multiple regions and the currentblock is predicted using the conventional inter prediction.

In the meantime, regarding the parent header, the parent headerincluding the information indicating whether or not the mixed interprediction is used may be transmitted by being included in a blockheader, a slice header, a tile header, a picture header, or a sequenceheader.

FIG. 23 is a diagram illustrating a process of decoding informationencoded in the manner shown in FIG. 21, namely, the informationindicating which type of inter prediction has been used for the block tobe currently encoded.

The decoding device 400 decodes the information indicating which type ofinter prediction has been used for the block to be currently encoded, atstep S1001, decodes the motion information at step S1002, and ends thealgorithm.

In the case where the information indicating whether or not the mixedinter prediction is used is present in the parent header of thebitstream, when the information indicating whether or not the mixedinter prediction is used indicates true, the information indicatingwhich type of inter prediction has been used for the block to becurrently encoded is present in the bitstream. When the informationindicating whether or not the mixed inter prediction is used indicatesfalse, the information indicating which type of inter prediction hasbeen used is not present within the bitstream, and in this case, thecurrent block is not partitioned into multiple regions and the currentblock is predicted using the conventional inter prediction.

The parent header including the information indicating whether or notthe mixed inter prediction is used may be a block header, a sliceheader, a tile header, a picture header, or a sequence header. In theparent header, the information indicating whether or not the mixed interprediction is used may be transmitted by being included in a blockheader, a slice header, a tile header, a picture header, or a sequenceheader.

FIG. 24 is a flowchart illustrating a method of generating an interprediction block by using the information indicating which type of interprediction has been used. The method shown in FIG. 24 may be performedby the decoding device 400.

First, it is determined at step S1101 whether or not the informationindicating which type of inter prediction has been used indicates use ofthe mixed inter prediction. When the mixed inter prediction is used forthe current block to be decoded, the current block is partitioned intomultiple regions at step S1102. For example, the current block may bepartitioned into the region A 500-a and the region B 500-b as shown inFIG. 5.

Here, it is possible that that the size of each region resulting fromthe partitioning is signaled from the encoding device 100 to thedecoding device 400 on a per-block basis or through the parent header oris set to a preset value.

Afterward, according to the method shown in FIG. 20, a motion vector ofa first region, for example, the region A 500-a, is estimated, and aprediction block is generated at step S1103.

Next, regarding a second region, for example, the region B 500-b, thedecoded motion vector is used to generate a prediction block at stepS1104, and the algorithm ends.

When the information indicating which type of inter prediction has beenused indicates that the mixed inter prediction is not used or when theinformation, included in the parent header, indicating whether or notthe mixed inter prediction is used indicates false, the conventionalinter prediction is applied as the prediction method of the currentblock 500. That is, the decoded motion information is used to generatethe prediction block of the current block 500 at step S1105, and thealgorithm ends. The size of the prediction block is the same as the sizeof the current block 500 to be decoded.

Fourth Exemplary Embodiment

Hereinafter, the fourth exemplary embodiment of the present inventionwill be described with reference to the drawings. The fourth exemplaryembodiment relates to a method to reduce blocking artifacts that mayoccur at the boundary of the block when the mixed inter predictionaccording to the third exemplary embodiment is performed.

FIG. 25 is a diagram illustrating a method of reducing blockingartifacts that may occur when the mixed inter prediction according tothe present invention is performed. Prediction block 1 and predictionblock 2 shown in FIG. 25 may correspond to the prediction block of theregion A 500-a and the prediction block of the region B 500-b shown inFIG. 18, respectively.

To summarize the fourth exemplary embodiment of the present invention,first, the regions positioned at the boundaries of the prediction blockare partitioned into sub-blocks in a predetermined size. Afterward, themotion information of the sub-block around the sub-block of theprediction block is applied to the sub-block of the prediction block sothat a new prediction block is generated. Afterward, a weighted sum ofthe sub-block of the prediction block and the new prediction block isobtained so that the final sub-block of the prediction block isgenerated. This is referred to as overlapped block motion compensation(OBMC).

Referring to FIG. 25, in the case of a sub-block P2 present in theprediction block 1 in FIG. 25, the motion information of the neighboringsub-block A2 is applied to the sub-block P2 to generate a new predictionblock of the sub-block P2, and then the weighted sum is applied as shownin FIG. 26 to generate the final sub prediction block.

For convenience of description, it is assumed that the size of eachsub-block shown in FIG. 25 is 4×4; that there are the prediction block1, eight sub-blocks A1 to A8 adjacent to the upper side thereof, andeight sub-blocks B1 to B8 adjacent to the left side thereof; and thatthere are the prediction block 2, four sub-blocks C1 to C4 adjacent tothe upper side thereof, and four sub-blocks D1 to D8 adjacent to theleft side thereof.

For convenience of description, although the horizontal and verticallengths of each sub-block are assumed to be four, other various valuesmay be encoded on a per-block basis or through the parent header and maybe then signaled to the decoding device 400. Accordingly, the encodingdevice 100 and the decoding device 400 may set the size of the sub-blockto be the same. Alternatively, it is possible that the encoding device100 and the decoding device 400 use sub-blocks in a preset same size.

FIG. 26 is a diagram illustrating a method of applying a weighted sum ofthe sub-block within the prediction block and the sub-block adjacent tothe upper side thereof.

Referring to FIG. 26, the final prediction pixel c is generated usingEquation below.

c=W1×a+(1−W1)×b  [Equation 5]

In addition to the prediction pixel c, the remaining 15 pixels may becomputed in a manner similar to the above. P2 to P8 in FIG. 13 arereplaced by new prediction pixels to which the weighted sum is appliedthrough the process shown in FIG. 26.

FIG. 27 is a diagram illustrating a method of applying a weighted sum ofa sub-block within a prediction block and a sub-block adjacent to theleft side thereof. The sub-blocks P9 to P15 are replaced by newprediction pixel values to which the weighted sum is applied as shown inFIG. 27.

Referring to FIG. 26, the same weighting factors are applied to thepixels on a per-row basis, and referring to FIG. 27, the same weightingfactors are applied to the pixels on a per-column basis.

In the case of the sub-block P1 in FIG. 25, the weighted sum with thepixels within the neighboring sub-block A1 is performed as shown in FIG.26, and then the weighted sum with the pixels within the sub-block B1 isperformed as shown in FIG. 27, thereby obtaining final predictionvalues.

Also in the case of the sub-blocks P16 to P22 present in the predictionblock 2 shown in FIG. 25, the weighted sum calculation method shown inFIG. 26 or FIG. 27 is used to obtain the final prediction values. Here,the neighboring sub-blocks used for the weighted sum are C1 to C4 or D1to D4.

In the meantime, not only the pixel values of the sub-blocks P16 to P22are replaced, but also the pixel values of the neighboring sub-blocks C1to C4, D1 to D4 may be replaced by new values through the weighted sumcalculation. For example, in the case of the sub-block C2, the motioninformation of the sub-block P17 is applied to the sub-block C2 togenerate a prediction sub-block, and then the pixel values within theprediction sub-block and the pixel values of the sub-block C2 aresubjected to the weighted sum so that the pixel values of the sub-blockC2 to which the weighted sum is applied is generated.

FIG. 28 is a flowchart illustrating a process of determining whether ornot the weighted sum is applied between sub-blocks at the boundary ofthe block, when the mixed inter prediction according to the presentinvention is performed.

The variable BEST_COST storing the optimum cost is initialized to themaximum value, COMBINE_MODE storing whether or not the mixed interprediction is used is initialized to false, and WEIGHTED_SUM storingwhether or not the weighted sum is used between sub-blocks isinitialized to false at step S1501. Afterward, inter prediction usingthe conventional method is performed, and then cost_A is computed atstep S1502. The mixed inter prediction is performed, and then cost_B iscomputed at step S1503. After comparing the two costs at step S1504,when the value of cost_A is lower, COMBINE_MODE is set to false toindicate that the mixed inter prediction is not used and BEST_COSTstores cost_A at step S1505.

When the value of cost_B is lower, COMBINE_MODE is set to true toindicate that the mixed inter prediction is used and BEST_COST storescost_B at step S1506. Afterward, the weighted sum is applied between thesub-blocks and cost_C is computed at step S1507. After comparingBEST_COST with cost_C at step S1508, when BEST_COST is lower thancost_C, the variable WEIGHTED_SUM is set to false to indicate that theweighted sum is not applied between the sub-blocks at step S1509.Otherwise, the variable WEIGHTED_SUM is set to true to indicate that theweighted sum is applied between the sub-blocks at step S1510 and thealgorithm ends.

FIG. 29 is a flowchart illustrating a process of encoding informationdetermined by the method in FIG. 27, namely, information indicatingwhether or not a weighted sum is applied between sub-blocks. The processshown in FIG. 29 may be performed by the image encoding device 100.First, the encoding device 100 encodes the information indicating whichtype of inter prediction has been used at step S1601, and encodes themotion information at step S1602. Afterward, the information indicatingwhether or not the weighted sum is applied between the sub-blocks isencoded at step S1603.

When the information indicating whether or not the mixed interprediction is used is present in the parent header of the bitstream, andwhen the information indicating whether or not the mixed interprediction is used indicates true, the information indicating whether ornot the weighted sum is applied between the sub-blocks is encoded andthen included in the bitstream. However, when the information, includedin the parent header, indicating whether or not the mixed interprediction is used indicates false, the information indicating whetheror not the weighted sum is applied between the sub-blocks is not presentwithin the bitstream.

FIG. 30 is a flowchart illustrating a process of decoding informationindicating whether or not a weighted sum is applied between sub-blocks.The process shown in FIG. 30 may be performed by the image decodingdevice 400. First, the decoding device 400 decodes the informationindicating which type of inter prediction has been used at step S1701,and decodes the motion information at step S1702. Afterward, theinformation indicating whether or not the weighted sum is appliedbetween the sub-blocks is decoded at step S1703.

When the information indicating whether or not the mixed interprediction is used is present in the parent header of the bitstream, andwhen the information indicating whether or not the mixed interprediction is used indicates true, the information indicating whether ornot the weighted sum is applied between the sub-blocks is encoded andthen included in the bitstream.

However, when the information, included in the parent header, indicatingwhether or not the mixed inter prediction is used indicates false, theinformation indicating whether or not the weighted sum is appliedbetween the sub-blocks is not present within the bitstream. In thiscase, it may be inferred that the information indicating whether or notthe weighted sum is applied between the sub-blocks indicates that theweighted sum is not applied between the sub-blocks.

Fifth Exemplary Embodiment

FIGS. 31a and 31b are diagrams illustrating inter prediction using areconstructed pixel region according to the fifth exemplary embodimentof the present invention. In the inter prediction using thereconstructed pixel region according to the present invention,particularly, the reconstructed pixel region may be used to derive themotion vector of the current block.

FIG. 31a shows a current block 252 to be encoded or decode and apre-reconstructed pixel region C 251 as a region adjacent to the currentblock 252. The reconstructed pixel region C 251 includes two regions,regions at the left side and the upper side of the current block 252.The current block 252 and the reconstructed pixel region C 251 areincluded within the current image 250. The current image 250 may be apicture, a slice, a tile, a coding tree block, a coding block, or otherimage regions. The reconstructed pixel region C 251 may correspond to areconstructed pixel region after being encoded before encoding of thecurrent block 252 in terms of encoding, and may correspond to apre-reconstructed pixel region before decoding of the current block 252in terms of decoding.

Before encoding or decoding of the current block, the reconstructedpixel region C 251 neighbors the current block 252, and thus the imageencoding device 100 and the image decoding device 400 may use the samereconstructed pixel region C 251. Therefore, without encoding the motioninformation of the current block 252 by the image encoding device 100,the reconstructed pixel region C 251 is used such that the imageencoding device 100 and the image decoding device 400 may generate themotion information of the current block 252 and the prediction block inthe same manner.

FIG. 31b shows an example of motion estimation and motion compensationusing a reconstructed pixel region. A reference image 253 shown in FIG.31b is searched for a region matched with the reconstructed pixel regionC 251 shown in FIG. 31a . When a reconstructed pixel region D 256 thatis most similar to the reconstructed pixel region C 251 is determined, adisplacement between a region 254, which is at the same position as thereconstructed pixel region C 251, and the reconstructed pixel region D256 is determined to be a motion vector 257 of the reconstructed pixelregion C 251. The motion vector 257 determined as described above isselected as the motion vector of the current block 252, and a predictionblock of the current block 252 may be derived using the motion vector257.

FIG. 32 is a diagram illustrating an example of a case where the motionvector 257 estimated as shown in FIG. 31b is set as an initial motionvector, the current block 252 is partitioned into multiple sub-blocks Ato D, and then motion estimation is further performed on a per-sub-blockbasis.

The sub-blocks A to D may be in an arbitrary size. MV_A to MV_D shown inFIG. 32 are the initial motion vectors of the sub-blocks A to D,respectively, and are the same as the motion vector 257 shown in FIG. 31b.

The size of each sub-block may be encoded on a per-block basis orthrough the parent header and may be transmitted to the decoding device400. Alternatively, it is possible that the encoding device 100 and thedecoding device 400 use the same preset size value of the sub-block.

In the meantime, as shown in FIGS. 7a to 7c , the reconstructed pixelregion C 251 may be in various shapes and/or sizes. Also, it is possiblethat the reconstructed pixel regions at the upper side and the left sideof the current block are used as the reconstructed pixel region C orthat as shown in FIGS. 31a to 31b , the two regions are combined into asingle piece to be used as the reconstructed pixel region C 251. Also,it is possible that the reconstructed pixel region C 251 is used bybeing subjected to subsampling.

Here, for convenience of description, the description is given assumingthat the reconstructed pixel region C 251 as shown in FIGS. 31a and 31bis used as a reconstructed pixel region.

FIG. 33 is a diagram illustrating an example in which the reconstructedpixel region C 251 and the current block are partitioned on aper-sub-block basis. Referring to FIG. 33, the reconstructed pixelregion C 251 is partitioned into sub-blocks a 285, b 286, c 287, and d288, and the current block is partitioned in to sub-blocks A 281, B 282,C 283, and D 284.

As the reconstructed pixel regions for the sub-block A 281, thesub-blocks a 285 and c 287 may be used. As the reconstructed pixelregions for the sub-block B 282, the sub-blocks b 286 and c 287 may beused. As the reconstructed pixel regions for the sub-block C 283, thesub-blocks a 285 and d 288 may be used. As the reconstructed pixelregions for the sub-block D 284, the sub-blocks b 286 and d 288 may beused.

FIG. 34 is a flowchart illustrating an example of an inter predictionmethod using a reconstructed pixel region. Referring to FIGS. 31a and31b , the reconstructed pixel region 251 of the current block 252 is setat step S291, and then the reconstructed pixel region 251 is used toperform motion estimation on the reference image 253 at step S292. Asthe result of the motion estimation, the motion vector 257 of thereconstructed pixel region 251 is obtained. Afterward, as shown in FIG.33, the reconstructed pixel region is set on a per-sub-block basis atstep S293, the motion vector 257 estimated at step S292 is set as astart point, and then motion is estimated on a per-sub-block basis ofthe current block at step S294.

FIG. 35 is a diagram illustrating an example of partitioning areconstructed pixel region into sub-blocks by using reconstructed blocksneighboring a current block according to the present invention.

According to the embodiment of the present invention, the reconstructedneighboring pixel region used for prediction of the current block may bepartitioned on the basis of a partitioning structure of reconstructedneighboring blocks. In other words, on the basis of at least one amongthe number of the reconstructed neighboring blocks, the sizes of thereconstructed neighboring blocks, the shapes of the reconstructedneighboring blocks, and the boundaries between the reconstructedneighboring blocks, the reconstructed pixel region may be partitioned.

Referring to FIG. 35, there are reconstructed block 1 2101 toreconstructed block 5 2105 around the current block 2100 to be encodedor decoded. When the reconstructed pixel region is set as shown in FIG.5a , efficiency in motion estimation may decrease due to the dramaticdifference in pixel values, which may be present at each of theboundaries of the reconstructed block 1 2101 to the reconstructed block5 2105. Therefore, as shown in FIG. 35, it may be efficient to partitionthe reconstructed pixel region into sub-blocks a to e for use. Dependingon how the pre-reconstructed blocks around the current block 2100 arepartitioned, the reconstructed pixel region shown in FIG. 35 may bepartitioned.

Specifically, the number of reconstructed neighboring blocks may beconsidered in partitioning of the reconstructed pixel region. Referringto FIG. 35, two reconstructed blocks, the reconstructed block 1 2101 andthe reconstructed block 2 2102, are present at the upper side of thecurrent block 2100. Three reconstructed blocks, the reconstructed block3 2103 to the reconstructed block 5 2105, are present at the left sideof the current block 2100. Considering this point, the reconstructedpixel region at the upper side of the current block 2100 is partitionedinto two sub-blocks, the sub-blocks a and b. The reconstructed pixelregion at the left side of the current block 2100 is partitioned intothree sub-blocks, the sub-blocks c to e.

Alternatively, the sizes of the reconstructed neighboring blocks may beconsidered in partitioning of the reconstructed pixel region. Forexample, the height of the sub-block c of the reconstructed pixel regionat the left side of the current block 2100 is the same as that of thereconstructed block 3 2103. The height of the sub-block d is the same asthat of the reconstructed block 4 2104. The height of the sub-block ecorresponds to a value obtained by subtracting the height of thesub-block c and the height of the sub-block d from the height of thecurrent block 2100.

Alternatively, the boundaries between the reconstructed neighboringblocks may be considered in partitioning of the reconstructed pixelregion. Considering the boundary between the reconstructed block 1 2101and the reconstructed block 2 2102 at the upper side of the currentblock 2100, the reconstructed pixel region at the upper side of thecurrent block 2100 is partitioned into two sub-blocks, the sub-blocks aand b. Considering the boundary between the reconstructed block 3 2103and the reconstructed block 4 2104 and the boundary between thereconstructed block 4 2104 and the reconstructed block 5 2105 at theleft side of the current block 2100, the reconstructed pixel region atthe left side of the current block 2100 is partitioned into threesub-blocks, the sub-blocks c to e.

In the meantime, there may be various conditions with respect to whichregion of the sub-blocks a to e is used to perform motion estimation.For example, it is possible that motion estimation is performed usingonly one reconstructed pixel region having the largest area, or it ispossible that m reconstructed pixel regions from the top and nreconstructed pixel regions from the left side are selected according tothe priority and used for motion estimation. Alternatively, it is alsopossible that a filter such as a low-pass filter is applied between thesub-blocks a to e to relieve the dramatic difference in pixel values andthen one reconstructed pixel region 251 as shown in FIG. 5a is used.

FIG. 36 is a diagram illustrating an example of partitioning a currentblock into multiple sub-blocks by using reconstructed blocks neighboringthe current block according to the present invention.

The method of partitioning the current block shown in FIG. 36 intomultiple sub-blocks is similar to the method of partitioning thereconstructed pixel region shown in FIG. 35. That is, the current blockto be encoded or decoded may be partitioned on the basis of apartitioning structure of reconstructed neighboring blocks. In otherwords, on the basis of at least one among the number of thereconstructed neighboring blocks, the sizes of the reconstructedneighboring blocks, the shapes of the reconstructed neighboring blocks,and the boundaries between the reconstructed neighboring blocks, thecurrent block may be partitioned.

The current block shown in FIG. 36 is partitioned into multiplesub-blocks A to F. Inter prediction may be performed on a per-sub-blockbasis, wherein the sub-blocks result from the partitioning. Here, interprediction may be performed using reconstructed regions a and c in FIG.10 for the sub-block A, using reconstructed regions b and c for thesub-block B, using reconstructed regions a and d for the sub-block C,using reconstructed regions b and d for the sub-block D, usingreconstructed regions a and e for the sub-block E, and usingreconstructed regions b and e for the sub-block F.

Alternatively, it is possible that priority is set depending on thesizes of the sub-blocks and the reconstructed pixel regions. Forexample, in the case of the sub-block A shown in FIG. 36, because thelength is longer than the height, it is possible that the reconstructedregion a has priority over the reconstructed region c and interprediction is performed only using the reconstructed region a.Alternatively, conversely, it is possible that the reconstructed regionc has priority depending on the situation, such as imagecharacteristics, and the like.

FIG. 37 is a flowchart illustrating a method of partitioning a currentblock into multiple sub-blocks according to an embodiment of the presentinvention.

Referring to FIG. 37, first, on the basis of the neighboring blocks ofthe current block to be encoded or decoded, the current block ispartitioned into multiple sub-blocks at step S2201. The neighboringblocks of the current block are pre-reconstructed blocks as shown inFIG. 36. As described above referring to FIG. 36, the current block tobe encoded or decoded may be partitioned on the basis of a partitioningstructure of reconstructed neighboring blocks. That is, on the basis ofat least one among the number of the reconstructed neighboring blocks,the sizes of the reconstructed neighboring blocks, the shapes of thereconstructed neighboring blocks, and the boundaries between thereconstructed neighboring blocks, the current block may be partitioned.

Next, multiple sub-blocks within the current block are encoded ordecoded at step S2203. According to the embodiment of the presentinvention, as described above, each of the sub-blocks A to F of thecurrent block shown in FIG. 36 may be encoded or decoded using interprediction. Here, inter prediction may be performed using reconstructedregions a and c in FIG. 35 for the sub-block A, using reconstructedregions b and c for the sub-block B, using reconstructed regions a and dfor the sub-block C, using reconstructed regions b and d for thesub-block D, using reconstructed regions a and e for the sub-block E,and using reconstructed regions b and e for the sub-block F. Informationrelated to inter prediction, such as sub_block information indicatingwhether or not partitioning into sub-blocks is performed, which isobtained by performing inter prediction on each of the sub-blocks A toF, motion information, or the like, may be encoded or decoded.

The method shown in FIG. 37 may be performed by the inter predictionmodule 103 of the image encoding device 100 or by the inter predictionmodule 408 of the image decoding device 400. The reference images usedin inter prediction are stored in the memory 112 of the image encodingdevice 100 or in the memory 406 of the image decoding device 400. Theinter prediction module 103 or the inter prediction module 408 maygenerate, with reference to the reference image stored in the memory 112or the memory 406, the prediction block of the current block 51.

FIG. 38 is a flowchart illustrating a method of partitioning areconstructed region used in encoding or decoding of a current blockinto multiple sub-blocks according to an embodiment of the presentinvention.

Referring to FIG. 13, first, on the basis of the neighboring blocks ofthe current block to be encoded or decoded, the pre-reconstructed pixelregion is partitioned into multiple sub-blocks at step S2211. Asdescribed above referring to FIG. 35 and/or FIG. 36, the reconstructedneighboring pixel region used for prediction of the current block may bepartitioned on the basis of a partitioning structure of reconstructedneighboring blocks. In other words, on the basis of at least one amongthe number of the reconstructed neighboring blocks, the sizes of thereconstructed neighboring blocks, the shapes of the reconstructedneighboring blocks, and the boundaries between the reconstructedneighboring blocks, the reconstructed pixel region may be partitioned.

Next, using at least one sub-block included in the reconstructed pixelregion, at least one among the multiple sub-blocks within the currentblock is encoded or decoded at step S2213. For example, as describedabove referring to FIG. 36, inter prediction may be performed usingreconstructed regions a and c in FIG. 35 for the sub-block A, usingreconstructed regions b and c for the sub-block B, using reconstructedregions a and d for the sub-block C, using reconstructed regions b and dfor the sub-block D, using reconstructed regions a and e for thesub-block E, and using reconstructed regions b and e for the sub-blockF. Information related to inter prediction, such as sub_blockinformation indicating whether or not partitioning into sub-blocks isperformed, which is obtained by performing inter prediction on each ofthe sub-blocks A to F, motion information, or the like, may be encodedor decoded.

The method shown in FIG. 38 may be performed by the inter predictionmodule 103 of the image encoding device 100 or by the inter predictionmodule 408 of the image decoding device 400. The reference images usedin inter prediction is stored in the memory 112 of the image encodingdevice 100 or in the memory 406 of the image decoding device 400. Theinter prediction module 103 or the inter prediction module 408 maygenerate, with reference to the reference image stored in the memory 112or in the memory 406, the prediction block of the current block 51.

FIG. 39 is a flowchart illustrating an example of an inter predictionmethod using the sub-blocks of the partitioned current block as shown inFIG. 36. The method shown in FIG. 39 may be performed by the interprediction module 103 of the image encoding device 100.

First, two variables used in this method, DMVD indication informationand SUB_BLOCK will be described. The decoder-side motion vectorderivation (DMVD) indication information or decoder-side motion vectorderivation indication information is information indicating whether theinter prediction using the conventional method is performed or theabove-described inter prediction using the reconstructed pixel regionaccording to the present invention is performed. When the DMVDindication information indicates false, it indicates that the interprediction using the conventional method is performed. When the DMVDindication information indicates true, it indicates that the interprediction using the reconstructed pixel region according to the presentinvention is performed.

The variable SUB_BLOCK indicates whether or not the current block ispartitioned into sub-blocks. When the value of SUB_BLOCK indicatesfalse, it indicates that the current block is not partitioned intosub-blocks. Conversely, when the value of SUB_BLOCK indicates true, itindicates that the current block is partitioned into sub-blocks.

Referring to FIG. 39, first, the variable DMVD indication information,which indicates whether or not the inter prediction using thereconstructed pixel region is performed, is set to false and thevariable SUB_BLOCK, which indicates whether or not partitioning intosub-blocks is performed, is set to false, and then inter prediction isperformed on the current block and cost_1 is computed at step S2301.

Afterward, SUB_BLOCK is set to true and inter prediction is performed,and then cost_2 is computed at step S2302. Next, the DMVD indicationinformation is set to true and SUB_BLOCK is set to false, and then interprediction is performed and cost_3 is computed at step S2303. Last, theDMVD indication information and SUB_BLOCK are set to true, and theninter prediction is performed and cost_4 is computed at step S2304. Thecalculated cost_1 to cost_4 are compared with each other, and then theoptimum inter prediction method is determined. The DMVD indicationinformation and the SUB_BLOCK information related to the determinedoptimum inter prediction method are stored, and then the algorithm ends.

FIG. 40 is a flowchart illustrating a method of encoding informationdetermined according to inter prediction shown in FIG. 39. The encodingmethod in FIG. 40 may be performed by the image encoding device 100.

In FIG. 36, the total number of sub-blocks of the current block is setto six, so that the variable BLOCK_NUM, which indicates the total numberof sub-blocks to be encoded, is initialized to six and the variableBLOCK_INDEX, which indicates the index of the sub-block to be encoded,is initialized to zero at step S2401. Here, the current block ispartitioned into the sub-blocks on the basis of the reconstructed blocksaround the current block, so that it is not necessary to encode thenumber of the sub-blocks. The image decoding device 400 partitions thecurrent block into the sub-blocks in the same manner as the imageencoding device 100, so that the image decoding device 400 is capable ofdetermining the number of the sub-blocks that may be present in thecurrent block.

After step S2401, SUB_BLOCK, the information indicating whether or notthe current block is partitioned into sub-blocks, is encoded at stepS2402. Whether or not the current block is partitioned into thesub-blocks is determined at step S2403, and when the partitioning intothe sub-blocks is not performed, the value of the variable BLOCK_NUM ischanged into one at step S2404.

Afterward, the DMVD indication information indicating whether or not theinter prediction using the reconstructed pixel region has been used isencoded at step S2405. Whether or not the inter prediction using thereconstructed pixel region has been used is determined at step S2406,and when the inter prediction using the reconstructed pixel region hasnot been used, the motion information is encoded at step S2407.Conversely, when the inter prediction using the reconstructed pixelregion has been used, the value of BLOCK_INDEX is increased at stepS2408 and is compared with the variable BLOCK_NUM at step S2409. Whenthe value of BLOCK_INDEX is the same as the value of BLOCK_NUM, thismeans that there is no more sub-block to be encoded in the currentblock, so that the algorithm ends. When the two values differ,proceeding to the subsequent sub-block to be encoded, which is presentwithin the current block, takes place and then the process repeats fromstep S2406.

FIG. 41 is a flowchart illustrating an example of a method of decodinginformation encoded by the encoding method shown in FIG. 40. In FIG. 36,the total number of sub-blocks of the current block is set to six, sothat the variable BLOCK_NUM, which indicates the total number ofsub-blocks to be decoded, is initialized to six and the variableBLOCK_INDEX, which indicates the index of the sub-block to be decoded,is initialized to zero at step S2501. As described above, on the basisof the reconstructed blocks around the current block, the image decodingdevice 400 and the image encoding device 100 partition the current blockinto the sub-blocks in the same manner, so that the informationindicating the number of the sub-blocks does not need to be transmittedto the image decoding device 400. The image decoding device 400 maydetermine by itself, on the basis of the reconstructed blocks around thecurrent block, the number of the sub-blocks that may be present in thecurrent block.

After step S2501, SUB_BLOCK, the information indicating whether or notthe current block is partitioned into sub-block, is decoded at stepS2502. Whether or not the current block is partitioned into thesub-blocks is determined at step S2403, and when the partitioning intothe sub-blocks is not performed, the value of the variable BLOCK_NUM ischanged into one at step S2404.

Afterward, the DMVD indication information indicating whether or not theinter prediction using the reconstructed pixel region has been used isdecoded at step S2505. Whether or not the inter prediction using thereconstructed pixel region has been used is determined at step S2506,and when the inter prediction using the reconstructed pixel region hasnot been used, the motion information is decoded at step S2507.Conversely, when the inter prediction using the reconstructed pixelregion has been used, the value of BLOCK_INDEX is increased at stepS2508 and is compared with the variable BLOCK_NUM at step S2509. Whenthe value of BLOCK_INDEX is the same as the value of BLOCK_NUM, thismeans that there is no more sub-block to be decoded in the currentblock, so that the algorithm ends. When the two values differ,proceeding to the subsequent sub-block to be decoded, which is presentwithin the current block, takes place and then the process repeats fromstep S2506.

Sixth Exemplary Embodiment

Hereinafter, the sixth exemplary embodiment of the present inventionwill be described with reference to the drawings.

FIGS. 42a and 42b are diagrams illustrating the sixth exemplaryembodiment of the present invention.

As shown in FIGS. 42a and 42b , assuming that reconstructed block 1 2601to reconstructed block 6 2606 are present around a current block 2600,the reconstructed pixel region may be partitioned into sub-blocks a to faccording to the method shown in FIG. 36. According to the method shownin FIG. 37, the current block 2600 may be partitioned into sub-blocks Ato I.

Here, the sub-blocks F, G, H, and I are spaced apart from thereconstructed pixel region rather than being in contact therewith, sothat inter prediction using the reconstructed pixel region may beinaccurate. Therefore, in the case of the sub-blocks F, G, H, and I, theconventional inter prediction is performed, and only in the case of thesub-blocks A to E, inter prediction using the reconstructed pixel regionmay be used.

When inter prediction using reconstructed pixel region is performed onthe sub-blocks A to E, inter prediction is performed using thereconstructed pixel region adjacent to each sub-block. For example,inter prediction may be performed using reconstructed pixel region b forthe sub-block B, using reconstructed pixel region c for the sub-block C,using reconstructed pixel region e for the sub-block D, and usingreconstructed pixel region f for the sub-block E. In the case of thesub-block A, according to preset priority, inter prediction may beperformed using either the reconstructed pixel region a or d, or usingthe reconstructed pixel regions a and d.

Alternatively, possibly, an index indicating which reconstructed pixelregion is used for each sub-block when inter prediction using thereconstructed pixel region is performed on the sub-blocks A to E, isencoded. For example, among the reconstructed pixel regions a to f, thereconstructed pixel region b may be used to perform inter prediction onthe sub-block A. In the case of the sub-block E, inter prediction may beperformed using the reconstructed pixel region c. In this case,according to the horizontal or vertical size of each of thereconstructed pixel regions a to f, the number of, the positions ofpixels in each region, and the like, the priority is determined andindexes are assigned.

In the case of the sub-blocks F to I, encoding or decoding may bepossible by performing the conventional inter prediction. Alternatively,as shown in FIG. 42b , the sub-blocks F to I may be integrated into oneand may be encoded or decoded using the conventional inter prediction.

FIG. 43 is a flowchart illustrating an example of a method ofdetermining an inter prediction mode according to the sixth exemplaryembodiment of the present invention described with reference to FIGS.42a and 42b . For convenience of description, in this example, as shownin FIG. 42b , it is assumed that as the neighboring reconstructedblocks, the reconstructed block 1 2601 to the reconstructed block 6 2606are present, and that the reconstructed pixel region is partitioned intosub-reconstructed pixel regions a to f. Further, it is assumed that thecurrent block 2600 is partitioned into the sub-blocks A to F. Here, thesub-block F is the one into which the sub-blocks F to I in FIG. 42a areintegrated.

Further, a case in which the index indicating which sub-reconstructedregion is used when performing inter prediction using the reconstructedpixel region is encoded, will be described as an example. As an example,it will be described that the sub-block F is encoded or decoded byperforming the conventional inter prediction. The description is givenassuming that among the sub-blocks within the current block, thesub-block F is encoded or decoded last.

Referring to FIG. 43, first, the current block is subjected to interprediction without being partitioned into sub-blocks, and then cost_1 iscomputed at step S2701. Afterward, the sub-blocks A to F are subjectedto inter prediction individually, and cost_A to cost_F are computed andthen added up to compute cost_2 at step S2702. The computed cost_1 iscompared with the computed cost_2 at step S2703. When cost_1 is lower,it is determined at step S2704 that partitioning into sub-blocks is notperformed. Otherwise, it is determined at step S2705 that partitioninginto sub-blocks is performed and inter prediction is performed, and thealgorithm ends.

FIG. 44 is a diagram illustrating a process of encoding informationdetermined by the method shown in FIG. 43. In FIG. 42b , the totalnumber of sub-blocks of the current block is set to six, so that thevariable BLOCK_NUM, which indicates the total number of sub-blocks to beencoded, is initialized to six and the variable BLOCK_INDEX, whichindicates the index of the sub-block to be encoded, is initialized tozero at step S2801. Here, the current block is partitioned into thesub-blocks on the basis of the reconstructed blocks around the currentblock, so that it is not necessary to encode the number of thesub-blocks. The image decoding device 400 partitions the current blockinto the sub-blocks in the same manner as the image encoding device 100,so that the image decoding device 400 is capable of determining thenumber of the sub-blocks that may be present in the current block.

After step S2801, SUB_BLOCK, the information indicating whether or notthe current block is partitioned into sub-blocks, is encoded at stepS2802. Whether or not the current block is partitioned into thesub-blocks is determined at step S2803, and when the partitioning intothe sub-blocks is not performed, the value of the variable BLOCK_NUM ischanged into one at step S2804.

Step S2805, at which the value of BLOCK_INDEX is compared with the valueof BLOCK_NUM−1, is the step of determining whether the conventionalinter prediction is used for the block or the inter prediction using thereconstructed pixel region is used for the block. When the two valuesare the same, it is the last block, namely, the sub-block subjected tothe conventional inter prediction, so that the motion information isencoded at step S2806. Otherwise, it is the sub-block subjected to theinter prediction using the reconstructed pixel region, so that the indexindicating which sub-reconstructed region is used is encoded at stepS2807. Alternatively, it is possible that this step is skipped and thesame reconstructed region determined in the encoding device and thedecoding device is used.

Afterward, the index of the sub-block is increased at step S2808, andBLOCK_NUM is compared with BLOCK_INDEX to determine whether or notencoding of all the sub-blocks present in the current block iscompleted, at step S2809. If not, proceeding to step S2805 takes placeand the algorithm continues.

FIG. 45 is a diagram illustrating a process of decoding informationencoded by the method shown in FIG. 44. In FIG. 42b , the total numberof sub-blocks of the current block is set to six, so that the variableBLOCK_NUM, which indicates the total number of sub-blocks to be encoded,is initialized to six and the variable BLOCK_INDEX, which indicates theindex of the sub-block to be encoded, is initialized to zero at stepS2901. Here, the current block is partitioned into the sub-blocks on thebasis of the reconstructed blocks around the current block, so that itis not necessary to encode the number of the sub-blocks. The imagedecoding device 400 partitions the current block into the sub-blocks inthe same manner as the image encoding device 100, so that the imagedecoding device 400 is capable of determining the number of thesub-blocks that may be present in the current block.

After step S2901, SUB_BLOCK, the information indicating whether or notthe current block is partitioned into sub-blocks, is decoded at stepS2902. Whether or not the current block is partitioned into thesub-blocks is determined at step S2903, and when the partitioning intothe sub-blocks is not performed, the value of the variable BLOCK_NUM ischanged into one at step S2904.

Step S2905, at which the value of BLOCK_INDEX is compared with the valueof BLOCK_NUM−1, is the step of determining whether the conventionalinter prediction is used for the block or the inter prediction using thereconstructed pixel region is used for the block. When the two valuesare the same, it is the last block, namely, the sub-block subjected tothe conventional inter prediction, so that the motion information isdecoded at step S2906. Otherwise, it is the sub-block subjected to theinter prediction using the reconstructed pixel region, so that the indexindicating which sub-reconstructed region is used is decoded at stepS2907. Alternatively, it is possible that this step is skipped and thesame reconstructed region determined in the encoding device and thedecoding device is used. Afterward, the index of the sub-block isincreased at step S2908, and BLOCK_NUM is compared with BLOCK_INDEX todetermine whether or not decoding of all the sub-blocks present in thecurrent block is completed, at step S2909. If not, proceeding to stepS2905 takes place and the algorithm continues.

Although the exemplary methods described in the present invention arerepresented as a series of operations for clarity of description, theorder of the steps is not limited thereto. When necessary, the steps maybe performed simultaneously or in a different order. In order to realizethe method according to the present invention, other steps may be addedto the illustrative steps, some steps may be excluded from theillustrative steps, or some steps may be excluded while additional stepsmay be included.

The various embodiments of the present invention are not intended tolist all possible combinations, but to illustrate representative aspectsof the present invention. The matters described in the variousembodiments may be applied independently or in a combination of two ormore.

Further, the various embodiments of the present invention may beimplemented by hardware, firmware, software, or combinations thereof. Inthe case of implementation by hardware, implementation is possible byone or more application specific integrated circuits (ASICs), digitalsignal processors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), general processors, controllers, micro controllers,microprocessors, or the like.

The scope of the present invention includes software ormachine-executable instructions (for example, an operating system, anapplication, firmware, a program, or the like) that cause operationaccording to the methods of the various embodiments to be performed on adevice or a computer, and includes a non-transitory computer-readablemedium storing such software or instructions to execute on a device or acomputer.

INDUSTRIAL APPLICABILITY

The present invention is applicable to a field of encoding or decodingan image signal.

1. An image decoding method comprising: selecting a reconstructed pixelregion within an image to which a current block to be decoded belongs;deriving, on the basis of the reconstructed pixel region and a referenceimage of the current block, a motion vector of the reconstructed pixelregion; and selecting the derived motion vector as a motion vector ofthe current block.
 2. The image decoding method of claim 1, wherein thederiving of the motion vector of the reconstructed pixel regionincludes: determining a region corresponding to the reconstructed pixelregion, within the reference image; and deriving, on the basis of aposition of the determined region corresponding to the reconstructedpixel region, the motion vector of the reconstructed pixel region. 3.The image decoding method of claim 1, further comprising: decodingdecoder-side motion vector derivation indication information, wherein onthe basis of the decoder-side motion vector derivation indicationinformation, the motion vector of the reconstructed pixel region isderived.
 4. An image encoding method comprising: selecting areconstructed pixel region within an image to which a current block tobe encoded belongs; deriving, on the basis of the reconstructed pixelregion and a reference image of the current block, a motion vector ofthe reconstructed pixel region; and selecting the derived motion vectoras a motion vector of the current block.
 5. The image encoding method ofclaim 4, wherein the deriving of the motion vector of the reconstructedpixel region includes: determining a region corresponding to thereconstructed pixel region, within the reference image; and deriving, onthe basis of a position of the determined region corresponding to thereconstructed pixel region, the motion vector of the reconstructed pixelregion.
 6. The image encoding method of claim 5, further comprising:encoding decoder-side motion vector derivation indication information,wherein the decoder-side motion vector derivation indication informationindicates whether or not the derived motion vector of the reconstructedpixel region is selected as the motion vector of the current block. 7.An image decoding method comprising: selecting at least onereconstructed pixel region within an image to which a current block tobe decoded using affine inter prediction belongs; deriving, on the basisof the at least one reconstructed pixel region and a reference image ofthe current block, a motion vector of the at least one reconstructedpixel region; and selecting the derived motion vector of the at leastone reconstructed pixel region as a motion vector of at least onecontrol point of the current block.
 8. The image decoding method ofclaim 7, wherein the at least one reconstructed pixel region is a regionadjacent to the at least one control point of the current block.
 9. Theimage decoding method of claim 7, wherein the deriving of the motionvector of the at least one reconstructed pixel region includes:determining a region corresponding to the at least one reconstructedpixel region, within the reference image; and deriving, on the basis ofa position of the determined region corresponding to the at least onereconstructed pixel region, the motion vector of the at least onereconstructed pixel region.
 10. An image decoding method comprising:partitioning a current block to be decoded into multiple regionsincluding a first region and a second region; obtaining a predictionblock of the first region; and obtaining a prediction block of thesecond region, wherein the prediction block of the first region and theprediction block of the second region are obtained by different interprediction methods.
 11. The image decoding method of claim 10, whereinthe first region is a region adjacent to a reconstructed image regionwithin an image to which the current block belongs, and the secondregion is a region that is not in contact with the reconstructed imageregion within the image to which the current block belongs.
 12. Theimage decoding method of claim 10, further comprising: decodinginformation that indicates which type of inter prediction is used,wherein when the information indicates to derive the prediction block ofthe first region and the prediction block of the second region by usingthe different inter prediction methods, the prediction block of thefirst region and the prediction block of the second region are derivedusing the different inter prediction methods.
 13. An image encodingmethod comprising: partitioning a current block to be encoded intomultiple regions including a first region and a second region; obtaininga prediction block of the first region; and obtaining a prediction blockof the second region, wherein the prediction block of the first regionand the prediction block of the second region are obtained by differentinter prediction methods.
 14. The image encoding method of claim 13,wherein the first region is a region adjacent to a pre-reconstructedimage region within an image to which the current block belongs, and thesecond region is a region that is not in contact with thepre-reconstructed image region within the image to which the currentblock belongs.
 15. The image encoding method of claim 13, furthercomprising: encoding information that indicates which type of interprediction is used, wherein the information is information indicatingwhether or not the prediction block of the first region and theprediction block of the second region are derived using the differentinter prediction methods.
 16. An image decoding method comprising:partitioning, on the basis of blocks around a current block to bedecoded, the current block into multiple sub-blocks; and decoding themultiple sub-blocks of the current block.
 17. The image decoding methodof claim 16, wherein the partitioning of the current block into themultiple sub-blocks is performed on the basis of a partitioningstructure of neighboring blocks of the current block.
 18. The imagedecoding method of claim 16, wherein the partitioning of the currentblock into the multiple sub-blocks is performed on the basis of at leastone among the number of neighboring blocks, a size of the neighboringblocks, a shape of the neighboring blocks, and a boundary between theneighboring blocks.
 19. The image decoding method of claim 16, furthercomprising: partitioning a pre-reconstructed pixel region, which is aregion neighbors the current block, on a per-sub-block basis, wherein atthe decoding of the multiple sub-blocks of the current block, at leastone of the multiple sub-blocks of the current block is decoded using atleast one sub-block included in the reconstructed pixel region.
 20. Theimage decoding method of claim 16, further comprising: decodinginformation that indicates whether or not partitioning into sub-blocksis performed, wherein the partitioning of the current block into themultiple sub-blocks is performed on the basis of the information thatindicates whether or not partitioning into the sub-block is performed.